Motion controlled remote controller

ABSTRACT

A handheld device includes a display having a viewable surface and operable to generate an image indicating a currently controlled remote device and a gesture database maintaining a plurality of remote command gestures. Each remote command gesture is defined by a motion of the device with respect to a first position of the handheld device. The device includes a gesture mapping database comprising a mapping of each of the remote command gestures to an associated command for controlling operation of the remote device and a motion detection module operable to detect motion of the handheld device within three dimensions and to identify components of the motion in relation to the viewable surface. The device includes a control module operable to track movement of the handheld device using the motion detection module, to compare the tracked movement against the remote command gestures to determine a matching gesture, and to identify the one of the commands corresponding to the matching gesture. The device also includes a wireless interface operable to transmit the identified command to a remote receiver for delivery to the remote device.

TECHNICAL FIELD

The present invention relates generally to portable devices and, moreparticularly, to portable devices with a motion interface.

BACKGROUND

The use of computing devices, such as cellular phones and personaldigital assistants (PDAs) has grown rapidly. Such devices provide manydifferent functions to users through different types of interfaces, suchas keypads and displays. Some computing devices utilize motion as aninterface by detecting tilt of the device by a user. Someimplementations of a motion interface involve tethering a computingdevice with fishing lines or carrying large magnetic tracking units thatrequire large amounts of power.

SUMMARY

In accordance with the present invention, a handheld device with motiona motion interface is provided.

In accordance with a particular embodiment, a handheld device includes adisplay having a viewable surface and operable to generate an imageindicating a currently controlled remote device and a gesture databasemaintaining a plurality of remote command gestures. Each remote commandgesture is defined by a motion of the device with respect to a firstposition of the handheld device. The device includes a gesture mappingdatabase comprising a mapping of each of the remote command gestures toan associated command for controlling operation of the remote device anda motion detection module operable to detect motion of the handhelddevice within three dimensions and to identify components of the motionin relation to the viewable surface. The device includes a controlmodule operable to track movement of the handheld device using themotion detection module, to compare the tracked movement against theremote command gestures to determine a matching gesture, and to identifythe one of the commands corresponding to the matching gesture. Thedevice also includes a wireless interface operable to transmit theidentified command to a remote receiver for delivery to the remotedevice.

In accordance with another embodiment, a method for remotely controllingdevices includes generating, on a viewable surface of a handheld device,an image indicating a currently controlled remote device and maintaininga gesture database comprising a plurality of remote command gestures.Each remote command gesture is defined by a motion of the device withrespect to a first position of the handheld device. The method includesmaintaining a gesture mapping database comprising a mapping of each ofthe remote command gestures to an associated command for controllingoperation of the remote device, tracking movement of the handheld devicein relation to the viewable surface, comparing the tracked movementagainst the remote command gestures to determine a matching gesture,identifying the one of the commands corresponding to the matchinggesture, and transmitting the identified command to a remote receiverfor delivery to the remote device.

Technical advantages of particular embodiments include the ability of ahandheld device to control various other local and remote devicesthrough motion input of the handheld device. For example, gestures ofthe handheld device may be used to communicate commands to a deviceselected for control. Accordingly, motion input of one handheld devicemay be used to control a plurality of other devices thus facilitatingcontrol of the other devices for a user.

Other technical advantages will be readily apparent to one skilled inthe art from the following figures, descriptions and claims. Moreover,while specific advantages have been enumerated above, variousembodiments may include all, some or none of the enumerated advantages.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of particular embodiments of theinvention and their advantages, reference is now made to the followingdescriptions, taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 illustrates a handheld device with motion interface capability,in accordance with a particular embodiment;

FIG. 2 illustrates a motion detector of the handheld device of FIG. 1,in accordance with a particular embodiment;

FIG. 3 illustrates the use of motion detector components of the handhelddevice of FIG. 1, in accordance with a particular embodiment;

FIG. 4 illustrates an example handheld device with motion detectioncapability, in accordance with a particular embodiment;

FIG. 5 illustrates an example of selection and amplification of adominant motion of a handheld device, in accordance with a particularembodiment;

FIG. 6 is a flowchart illustrating preferred motion selection, inaccordance with a particular embodiment;

FIG. 7 is a flowchart illustrating the setting of a zero-point for ahandheld device, in accordance with a particular embodiment;

FIG. 8 illustrates an example of scrubbing functionality with a handhelddevice for virtual desktop navigation, in accordance with a particularembodiment;

FIG. 9 is a flowchart illustrating the scrubbing process of FIG. 8, inaccordance with a particular embodiment;

FIG. 10A illustrates an example of menu navigation using gesture input,in accordance with a particular embodiment;

FIG. 10B illustrates example gestures which may be used to performvarious functions at a handheld device, in accordance with a particularembodiment;

FIG. 11 illustrates an example of map navigation using motion input, inaccordance with a particular embodiment;

FIG. 12A illustrates a form of motion input cursor navigation, inaccordance with a particular embodiment;

FIG. 12B illustrates another form of motion input cursor navigation, inaccordance with a particular embodiment;

FIG. 13 is a flowchart illustrating a process for utilizing feedback inresponse to motion input, in accordance with a particular embodiment;

FIG. 14 illustrates an example system utilizing spatial signatures witha handheld device, in accordance with a particular embodiment;

FIG. 15 illustrates an example system in which motion input of ahandheld device controls multiple other devices, in accordance with aparticular embodiment;

FIG. 16 is a flowchart illustrating an environmental modeling process ofa handheld device, in accordance with a particular embodiment;

FIG. 17 illustrates example gestures which may be mapped to differentfunctions of a handheld device, in accordance with a particularembodiment;

FIG. 18 is a flowchart illustrating the utilization of a preexistingsymbol gesture, in accordance with a particular embodiment;

FIG. 19 is a flowchart illustrating the use of context-based gesturemapping, in accordance with a particular embodiment;

FIG. 20 is a flowchart illustrating the use of user-based gesturemapping, in accordance with a particular embodiment;

FIG. 21 is a flowchart illustrating the assignment process foruser-created gestures, in accordance with a particular embodiment;

FIG. 22 illustrates three gestures input using a handheld device withvarying levels of precision, in accordance with a particular embodiment;and

FIG. 23 is a flowchart illustrating a gesture recognition processutilizing a number of features, in accordance with a particularembodiment.

DETAILED DESCRIPTION

FIG. 1 illustrates a handheld device 10 with motion interfacecapability, in accordance with a particular embodiment of the presentinvention. Handheld device 10 can recognize movement of the device andcan perform various functions corresponding to such movement. Thus,movement of the device operates as a form of input for the device. Suchmovement input may directly alter what is being displayed on a devicedisplay or may perform other functions. Handheld device 10 may comprisea mobile phone, personal digital assistant (PDA), still camera, videocamera, pocket calculator, portable radio or other music or videoplayer, digital thermometer, game device, portable electronic device,watch or any other device capable of being held or worn by a user. Asindicated in the examples listed above, handheld device 10 may includewearable portable devices such as watches as well. A watch may includeany computing device worn around a user's wrist.

Handheld device 10 includes a display 12, input 14, processor 16, memory18, communications interface 20 and motion detector 22. Display 12presents visual output of the device and may comprise a liquid crystaldisplay (LCD), a light emitting diode (LED) or any other type of displayfor communicating output to a user. Input 14 provides an interface for auser to communicate input to the device. Input 14 may comprise akeyboard, keypad, track wheel, knob, touchpad, stencil or any othercomponent through which a user may communicate an input to device 10. Inparticular embodiments, display 12 and input 14 may be combined into thesame component, such as a touchscreen.

Processor 16 may be a microprocessor, controller or any other suitablecomputing device or resource. Processor 16 is adapted to execute varioustypes of computer instructions in various computer languages forimplementing functions available within system handheld device 10.Processor 16 may include any suitable controllers for controlling themanagement and operation of handheld device 10.

Memory 18 may be any form of volatile or nonvolatile memory including,without limitation, magnetic media, optical media, random access memory(RAM), read only memory (ROM), removable media or any other suitablelocal or remote memory component. Memory 18 includes components, logicmodules or software executable by processor 16. Memory 18 may includevarious applications 19 with user interfaces utilizing motion input,such as mapping, calendar and file management applications, as furtherdiscussed below. Memory 18 may also include various databases, such asgesture databases and function or gesture mapping databases, as furtherdiscussed below. Components of memory 18 may be combined and/or dividedfor processing according to particular needs or desires within the scopeof the present invention. Communications interface 20 supports wirelessor wireline communication of data and information with other devices,such as other handheld devices, or components.

Motion detector 22 tracks movement of handheld device 10 which may beused as a form of input to perform certain functions. Such inputmovement may result from a user moving the device in a desired fashionto perform desired tasks, as further discussed below.

It should be understood that handheld device 10 in accordance withparticular embodiments may include any suitable processing and/or memorymodules for performing the functions as described herein, such as acontrol module, a motion tracking module, a video analysis module, amotion response module, a display control module and a signaturedetection module.

In particular embodiments, input movement may be in the form oftranslation and/or gestures. Translation-based input focuses on abeginning point and endpoint of a motion and differences between suchbeginning points and endpoints. Gesture-based input focuses on an actualpath traveled by the device and is a holistic view of a set of pointstraversed. As an example, when navigating a map using translation-basedinput, motion in the form of an “O” may change the display during themovement but may ultimately yield no change between the informationdisplayed prior to the movement and the information displayed at the endof the movement since the device presumably will be in the same point asit started when the motion ends. However, in a gesture input mode thedevice will recognize that it has traveled in the form of an “O” becausein gesture-based input the device focuses on the path traveled duringthe motion or movement between a beginning point and an endpoint of thegesture (e.g., even though the beginning and endpoints may be the same).This gesture “O” movement may be mapped to particular functions suchthat when the device recognizes it has traveled along a path toconstitute an “O” gesture, it may perform the functions, as furtherelaborated upon below. In particular embodiments, movement of the deviceintended as a gesture may be recognized as by the device as a gesture bymatching a series, sequence or pattern of accelerations of the movementto those defining gestures of a gesture database.

Handheld devices in accordance with other embodiments may not includesome of the components of the device illustrated in FIG. 1. For example,some embodiments may include a handheld device 10 without an input 14separate from a motion detector such that motion of the device providesthe sole or primary input for the device. It should be noted thathandheld devices in accordance with other embodiments may includeadditional components not specifically illustrated with respect todevice 10.

FIG. 2 illustrates motion detector 22 of FIG. 1, in accordance with aparticular embodiment of the present invention. In this embodiment,motion detector 22 includes accelerometers 24 a, 24 b and 24 c; cameras26 a, 26 b and 26 c; gyros 28 a, 28 b and 28 c; rangefinders 30 a, 30 band 30 c; and a processor 32.

Accelerometers 24 a, 24 b and 24 c detect movement of the device bydetecting acceleration along a respective sensing axis. A particularmovement of the device may comprise a series, sequence or pattern ofaccelerations detected by the accelerometers. When the handheld deviceis tilted along a sensing axis of a particular accelerometer, thegravitational acceleration along the sensing axis changes. This changein gravitational acceleration is detected by the accelerometer andreflects the tilt of the device. Similarly, translation of the handhelddevice, or movement of the device without rotation or tilt also producesa change in acceleration along a sensing axis which is also detected bythe accelerometers.

In the illustrated embodiment, accelerometer 24 a comprises an x-axisaccelerometer that detects movement of the device along an x-axis,accelerometer 24 bcomprises a y-axis accelerometer that detects movementof the device along a y-axis and accelerometer 24 c comprises a z-axisaccelerometer that detects movement of the device along a z-axis. Incombination, accelerometers 24 a, 24 b and 24 c are able to detectrotation and translation of device 10. As indicated above, rotationand/or translation of device 10 may serve as an input from a user tooperate the device.

The use of three accelerometers for motion detection provides certainadvantages. For example, if only two accelerometers were used, themotion detector may not be able to disambiguate translation of thehandheld device from tilt in the plane of translation. However, using athird, z-axis accelerometer (an accelerometer with a sensing axis atleast approximately perpendicular to the sensing axes of the other twoaccelerometers) enables many cases of tilt to be disambiguated from manycases of translation.

It should be understood that some unique movements may exist that maynot be discernible from each other by accelerometers 24 a, 24 b and 24c. For example, movement comprising a certain rotation and a certaintranslation may appear to accelerometers 24 a, 24 b and 24 c as the samemovement as a different movement that comprises a different particularrotation and a different particular translation. If a motion detector 22merely included three accelerometers to detect movement (without anyadditional components to ensure greater accuracy), some unique,undiscernible movements may be mapped to the same function or may not bemapped to a function to avoid confusion.

As indicated above, motion detector 22 also includes cameras 26 a, 26 band 26 c, which may comprise charge coupled device (CCD) cameras orother optical sensors. Cameras 26 a, 26 b and 26 c provide another wayto detect movement of the handheld device (both tilt and translation).If only one camera were installed on a device for movement detection,tilt of the device may be indistinguishable from translation (withoutusing other motion detection components, such as accelerometers).However, by using at least two cameras, tilt and translation may bedistinguished from each other. For example, if two cameras wereinstalled on handheld device 10 (one on the top of the device and one onthe bottom of the device), each camera would see the world moving to theright when the device was translated to the left. If the device is lyinghorizontally and is rotated by lifting its left edge while lowering itsright edge, the camera on the bottom will perceive the world as movingto the right while the camera on the top will perceive the world asmoving to the left. Thus, when a device is translated, cameras onopposite surfaces will see the world move in the same direction (left inthe example given). When a device is rotated, cameras on oppositesurfaces will see the world move in opposite directions. This deductiveprocess can be reversed. If both cameras see the world moving in thesame direction, then the motion detector knows that the device is beingtranslated. If both cameras see the world moving in opposite directions,then the motion detector knows that the device is being rotated.

When the device is rotated, the magnitude of the movement of the worldto the cameras is directly related to the magnitude of the rotation ofthe device. Thus, the amount of the rotation can accurately bedetermined based on such movement of the world to the cameras. However,when the device is translated, the magnitude of the translation isrelated to both the magnitude of the movement of the world to thecameras and to the distance to the objects in the field of view of thecameras. Therefore, to accurately determine amount of translation usingcameras alone, some form of information concerning the distance toobjects in the camera fields of view must be obtained. However, in someembodiments cameras with rangefinding capability may be used.

It should be understood that even without distance information, opticalinformation can be of significant value when correlated against theinformation from accelerometers or other sensors. For example, opticalcamera input may be used to inform the device that no significant motionis taking place. This could provide a solution to problems of driftwhich may be inherent in using acceleration data to determine absoluteposition information for certain device functions.

As discussed above, distance information may be useful to determineamount of translation when cameras are being used to detect movement. Inthe illustrated embodiment, such distance information is provided byrangefinders 30 a, 30 b and 30 c. Rangefinders 30 a, 30 b and 30 c maycomprise ultrasound rangefinders, laser rangefinders or any othersuitable distance measuring component. Other components may also be usedto determine distance information. For example, cameras withrangefinding capability may be used, and multiple cameras may beutilized on the same side of the device to function as a range-finderusing stereopsis. Determined distance information allows for accurateand explicit computation of the portion of any apparent translation thatis due to translation and the portion that is due to rotation.

As indicated above, motion detector 22 additionally includes gyros 28 a,28 band 28 c. Gyros 28 a, 28 b and 28 c are used in combination with theother components of motion detector 22 to provide increased accuracy indetecting movement of device 10.

Processor 32 processes data from accelerometers 24, cameras 26, gyros 28and rangefinders 30 to produce an output indicative of the motion ofdevice 10. Processor 32 may comprise a microprocessor, controller or anyother suitable computing device or resource, such as a video analysismodule for receiving a video stream from each camera. In someembodiments, the processing described herein with respect to processor32 of motion detector 22 may be performed by processor 16 of handhelddevice 10 or any other suitable processor, including processors locatedremote to the device.

As discussed above, motion detector 22 includes three accelerometers,three cameras, three gyros and three rangefinders. Motion detectors inaccordance with other embodiments may include fewer or differentcomponents than motion detector 22. For example, some embodiments mayinclude a motion detector with three accelerometers and no cameras,gyros or rangefinders; two or three accelerometers and one or moregyros; two or three accelerometers and one or more cameras; or two orthree accelerometers and one or more rangefinders. In addition, thelocation of the motion detection components on the device may vary ondifferent embodiments. For example, some embodiments may include camerason different surfaces of a device while other embodiments may includetwo cameras on the same surface (e.g., to add rangefindingfunctionality).

Altering the type, number and location of components of motion detector22 may affect the ability of motion detector to detect or accuratelymeasure various types of movement. As indicated above, the type andnumber of components of motion detectors may vary in differentembodiments in order to fulfill particular needs. Fewer or less accuratecomponents may be used in particular embodiments when it is desired tosacrifice accuracy to reduce manufacturing cost of a handheld devicewith motion detection capabilities. For example, some handheld devicesmay only need to detect that the device has been translated and may notneed to detect exact amount of such translation to perform desiredfunctions of the device. Such handheld devices may thus include a motiondetector with cameras and without any sort of rangefinder or othercomponent providing distance information. In particular embodiments,components described above, such as cameras and rangefinders, may alsobe used for other purposes by the device than those described aboverelating to motion detection functionality.

FIG. 3 is a diagram illustrating the use of the motion detectorcomponents of handheld device 10 of FIG. 1. Raw data from motiondetection components is processed at processor 32. Such raw dataincludes x-axis accelerometer raw data 23 a, y-axis accelerometer rawdata 23 b and z-axis accelerometer raw data 23 c from accelerometers 24a, 24 b and 24 c, respectfully; camera raw data 25 a, camera raw data 25b and camera raw data 25 c from cameras 26 a, 26 b and 26 c,respectfully; gyro raw data 27 a, gyro raw data 27 b and gyro raw data27 c from gyros 28 a, 28 b and 28 c; and rangefinder raw data 29 a,rangefinder raw data 29 b and rangefinder 29 c from rangefinders 30 a,30 b and 30 c, respectively. If the handheld device includes more, feweror different motion detection components as may be the case in someembodiments, the raw data may correspond to the components included.

The raw data is processed at processor 32 to produce motion detectoroutput 34 identifying movement of device 10. In the illustratedembodiment, motion detector output 34 comprises translation along x, yand z axes and rotation with respect to the x, y and z axes. The motiondetector output is communicated to a processor 16 of the handheld devicewhich identifies the operation, function or task the device shouldperform (i.e., device behavior 36) based on the device motion. Theperformance of certain operations, functions or tasks based onparticular movements is further discussed below.

FIG. 4 is an isometric illustration of an example handheld device 31with motion detection capability, in accordance with particularembodiments. Handheld device 31 includes an x-axis accelerometer 33, ay-axis accelerometer 35 and a camera 37 oriented towards the z-axis.X-axis 38, y-axis 39 and z-axis 40 are also illustrated with respect todevice 31 for reference. Handheld device 31 may detect movement,including tilt and translation in various directions, usingaccelerometers 33 and 35 and camera 37. Handheld device 31 may alsoinclude other components, such as components illustrated and describedabove with respect to handheld device 10, such as display 12, input 14,processor 16, memory 18 and communications interface 20. As indicatedabove, particular embodiments may include handheld devices havingvarious types of motion detection components (including accelerometers,gyros, cameras, rangefinders or other suitable components) in anycombination and positioned or oriented in any suitable manner upon thedevices.

In particular embodiments, a user interface function may utilize inputmotion along one axis of motion at a time. For example, a deviceapplication may allow a user to scroll through a list displayed on thehandheld device by moving the device along a particular axis (e.g., inone direction or in two opposite directions). It may be very difficultfor a user to constrain the motion of the device to that particular axisas desired. In other words, some user generated device rotation ormovement along another axis may be difficult to avoid. To counter thisproblem, the device may include preferred motion selection including theselection and amplification of a dominant motion and the minimization ofmovement in other directions or axes.

FIG. 5 illustrates an example of selection and amplification of adominant motion and minimization of movement in another direction asdiscussed above. In the illustrated example, actual motion 41 representsmovement of a handheld device. Actual motion 41 comprises movement 42along one axis 44 and movement 46 along another axis 48 perpendicular toaxis 44. Since the amount of movement 42 is greater than the amount ofmovement 46, the handheld device may select movement 42 as the dominantmotion. The handheld device may then amplify this dominant motion andminimize movement 46 (the other motion) such that actual motion 41 istreated by the device as represented motion 50. The amount or size ofamplification of the dominant motion may vary in various embodimentsaccording to particular factors, such as the particular applicationbeing run on the device at the time. Moreover, amplification of adominant motion may also be based on magnitude of acceleration, speed ofmotion, ratio of a motion in one direction (e.g., movement 42) to motionin another direction (e.g., movement 46), size of underlying desktopbeing navigated or user preferences. In some embodiments, a handhelddevice may implement preferred motion selection only when certain motioncharacteristics occur. For example, in some cases the handheld devicemay select and amplify a dominant motion if motion in one axis is morethan two times greater than any other motion. The other, smaller motionmay then be minimized.

The selection and amplification of the dominant motion and minimizationof other motion may further expand a user's ability to take advantage ofmotion user interfaces and may also allow the handheld device, orapplications running on the device, to filter out undesired,user-induced noise. With this capability, the user may be able to, forexample, move the device left to pick a list to examine, then scrollthat list by moving up and down. Motion along inappropriate axes may beignored or substantially reduced by the device.

In particular embodiments, the selection and amplification of a dominantmotion and minimization of other motion may also be applied torotational motion of the device. Dominant motion around an axis may beselected and amplified in the same manner as motion along an axis asdescribed above with respect to translational motion. Moreover, rotationaround another axis (that is not dominant rotation) may be minimized.

FIG. 6 illustrates a preferred motion selection flowchart 60, inaccordance with a particular embodiment of the present invention. Inflowchart 60, raw data corresponding to movement of a handheld device isreceived. In the illustrated embodiment, the movement raw data includesx-acceleration data 62 a, y-acceleration data 62 b and z-accelerationdata 62 c that is processed at step 64 to yield an output indicatingmovement of the device. Other embodiments may include other types ofmovement raw data, such as optical or camera data, gyro data and/orrangefinder data. After the processing of the raw acceleration data 62,a dominant axis of motion is selected at step 66. If the selecteddominant axis of motion is the x-axis, then the movement along thex-axis is augmented at step 68 a. If the selected dominant axis ofmotion is the y-axis, then the movement along the y-axis is augmented atstep 68 b. If the selected dominant axis of motion is the z-axis, thenthe movement along the z-axis is augmented at step 68 c. The amount ofaugmentation of movement in the dominant axis of motion may vary indifferent embodiments according to the application being utilized orother characteristics. In some embodiments, user preferences 69 may beutilized to determine type or amount of movement augmentation. Movementalong axes other than the dominant axis of motion may be minimized asdiscussed above, such that such movement is ignored by a particularapplication in use. At step 70, the augmented movement is processed toyield device behavior 72. This processing step may include accessing anapplication being used to determine the particular device behavior toperform based on the augmented movement. Augmented movement may yieldany of a number of types of device behavior according to the applicationin use, a particular user or otherwise.

For particular user interfaces utilizing motion input, there may bevalue in having the position of a virtual display, or the informationdisplayed at display 12 of handheld device 10, linked to the position ofthe device. For example, in particular embodiments usingtranslation-based input such as for navigating a map displayed at thedevice, the position of the handheld device may directly determine theportion of the map displayed at display 12. However, if device positioninformation is kept in absolute terms (e.g., as with global positioningsatellite (GPS) based systems) the utility for many tasks such as map ormenu navigation may be impaired. Thus, it is beneficial in certaincircumstances to define a “zero point,” or an origin in a local context,that may be used to determine the behavior of the device. For example,if a zero point is defined when the device is at a point A, then motionbetween point A and a point B may be used as input. Particularly usefulapplications of setting a zero point may include external behaviors suchas moving the virtual display or locating applications in the spacearound a user's body. Setting a zero point also addresses internalbehaviors such as instructing the device to ignore the gravitationalacceleration at the current orientation to allow the device to act onlyon additional, and presumably user generated, accelerations.

Handheld devices according to particular embodiments may includeapplication user interfaces that utilize motion input only at certaintimes. At other times, for example, the motion of the device may not beutilized as input, and it may be useful to disengage or “turn off”motion sensitivity or the motion detection capability of the device.Disengagement of motion sensitivity may comprise, for example,deactivation of motion detector 22 of device 10 or other component, suchas a motion response module of the device. Particular embodiments thusallow for the selective engagement and disengagement of the motionsensitivity of the device.

As an example, a motion response module which modifies display 12 basedon motion detected at motion detector 22, may have a mode of operationin which it awaits a trigger for switching to another mode of operationin which motion sensitivity is enabled. When motion sensitivity is notenabled, any motion of the device may be disregarded. The trigger mayalso set a zero-point for the device. When the zero-point is set, themotion response module may measure a baseline orientation of the devicebased on measurement from motion detection components. The baselineorientation may comprise the position of the device (determined frominformation from motion detector components) when the trigger isreceived. Future movement of the device will be compared against thebaseline orientation to determine the functions to perform or themodifications which should be made at display 12 based on the user'smotion of the device.

Particular embodiments provide for any number of user-initiated actionsto act as a single trigger for zero-point selection and selectiveengagement/disengagement of the motion sensitivity of the device. Suchactions may include, for example, the pressing of a key on input 14,moving device 10 in a particular way (e.g., movement corresponding to aparticular gesture), and tapping display 12. It should be understoodthat any user-initiated action may set a zero-point and engage motionsensitivity of the device at the same time.

In some embodiments, a period of inactivity or minimal activity (i.e.,relative stillness) may also set a zero-point and engage or disengagemotion sensitivity. FIG. 7 illustrates a flowchart 80 for the passivesetting of a zero-point for a handheld device. Change in accelerationwith respect to an x-axis is detected at step 82 a, change inacceleration with respect to a y-axis is detected at step 82 b andchange in acceleration with respect to a z-axis is detected at step 82c. At steps 84 a, 84 b and 84 c, it is determined whether anyacceleration change detected is greater than a particular respectivethreshold. If detected acceleration change along each of the three axesis not greater than a set threshold, then the device may be consideredat rest, and at step 86 a zero-point will be set. An at rest positionmay be determined, for example, from stabilization of the raw data ormotion components of components of motion detector 22. However, ifdetected acceleration change along any of the three axes is greater thana set threshold, then the process returns to acceleration changedetection at steps 82. Thus, this method of passively setting azero-point may ensure that when the handheld device is at rest, a zeropoint will be set. Moreover, if a device is in constant motion but notbeing moved by a user at a particular time (e.g., resting in a trainmoving at a constant speed), a zero-point will be set since there willbe no detected change in acceleration. The use of thresholds todetermine whether an acceleration change is high enough so as not totrigger the setting of a zero-point enables a user to hold the devicestill to passively set the zero-point. Otherwise, this may be difficultsince a device with highly sensitive accelerometers may detectacceleration change as a result of very minor unintended, user-initiatedmovement. It should be understood that similar methods may be used inconnection with motion detectors with components other thanaccelerometers. Thresholds may also be used in such similar methods toaccount for small, unintended movements that may otherwise preventsetting of a zero point.

Particular embodiments of the present invention include the ability toallow a user to repeatedly selectively engage and disengage the motionsensitivity of the handheld device in order to allow greater movementthrough a virtual desktop (or information space) using motion input in alimited amount of physical space. This process can be analogized to“scrubbing” with a mouse controlling a cursor, or lifting the mouse offof a surface and replacing the mouse on the surface at a differentlocation to allow greater movement of the cursor. Lifting the mousebreaks the connection between the motion of the mouse and the motion ofthe cursor. Similarly, a user may be able to engage and disengage theconnection between the motion of a handheld device, such as device 10,and the operations, functions or actions based on movement of thedevice.

FIG. 8 illustrates an example of the use of scrubbing functionality tonavigate across a virtual desktop, or information space, larger than thedisplay of a handheld device. In the illustrated example, a handhelddevice is used to navigate through virtual desktop 90. Virtual desktop90 is illustrated as a grid map and may represent any suitableinformation through which a user may desire to navigate. Information ofvirtual desktop displayed at the handheld device is represented by box92. In this embodiment, translation of the handheld device is used tonavigate through virtual desktop 90. For example, a user may move thehandheld device from right to left to navigate from right to leftthrough information of virtual desktop 90. It should be understood thatwhile the example illustrated describes moving the device to the rightto implement the scrubbing process, it should be understood thathandheld devices of particular embodiments may be moved in any suitablemanner to implement the scrubbing process.

As indicated above, box 92 represents the information of virtual desktop90 currently displayed at the device. If a user desired to viewinformation represented at box 94, the user may move the handheld devicefrom left to right. For purposes of this example, imagine that the usermoves the device to the right, and the information of virtual desktop 90contained in box 94 is displayed at the device. Also imagine that theuser's arm is now outstretched to the user's right such that the usermust walk or otherwise move further right in order to view at thedisplay of the device information of virtual desktop 90 that is to theright of box 94. If this were the case, and the user could not, or didnot desire to, walk or otherwise move further to the right so that thedevice would be moved further to the right, then the user couldselectively disengage the motion sensitivity of the handheld device,move the device back to the left, selectively reengage the motionsensitivity of the device and move the device back to the right todisplay information to the right of box 94. In this manner, the usercould display information of virtual desktop 90 contained in box 96, andthis process could be repeated to display information contained in box98, which is further to the right of box 96.

The selective disengagement and reengagement of the motion sensitivityof the device in order to allow greater movement within a virtualdesktop in a limited amount of physical space may be enabled in any of avariety of ways, such as by a key on an input of the device, moving thedevice according to a particular gesture or movement (e.g., an arcmovement) or tapping the device display. Any other user-initiated actionmay be used to disengage and reengage motion sensitivity for thispurpose. Particular embodiments may allow multiple actions to disengageand reengage motion sensitivity of the device. Moreover, a user actionthat disengages motion sensitivity of the device may be different from auser action that reengages motion sensitivity. This scrubbing processmay be performed in any suitable application, such as map navigation,menu navigation and scrolling through a list.

FIG. 9 is a flowchart illustrating the steps of the scrubbing processdescribed above with respect to FIG. 8, in accordance with a particularembodiment. The flowchart begins at step 100, where the handheld deviceis moved to the right to go from displaying information of box 92 ofvirtual display 90 to information of box 94. As indicated above, theuser may desire to display information further to the right of box 94but may have run out of physical space to move the device further to theright. Thus, the user disengages the motion sensitivity of the device atstep 102. Any suitable user action may perform such disengagement, suchas the pressing of a button on the device or moving the device accordingto a particular gesture. At step 104, the user moves the device to theleft so that the user will have more physical space through which theuser may move the device to the right when motion sensitivity isreengaged.

At step 106, the user reengages motion sensitivity of the device. Again,such reengagement may be performed by any suitable user action, and suchuser action may be different from the user action performed to disengagemotion sensitivity in step 102. Since motion sensitivity has beenreengaged, the user moves the device to the right at step 108 in orderto change the information being displayed at the device from theinformation of box 94 to the information of box 96. At step 110, it isdetermined whether further movement of the device to the right isneeded. If further movement is needed (e.g., to display the informationof virtual display 90 in box 98), then the process returns to step 102where motion sensitivity of the device is again disengaged. If nofurther movement is needed, then the process ends. As indicated above,this scrubbing process may be utilized in any suitable application ofthe device that supports motion input, and the device may be moved inany suitable manner in order to implement this functionality.

As indicated above, a particular movement of the device (e.g., aparticular gesture) may be utilized in the scrubbing process to signalto the device not to change the information presented on the displayduring such movement. This allows the user to return the device to aposition from which the user may move the device to further change theinformation presented on the display. For example, the device may be ata base reference position from which movement of the device changes theinformation displayed. A particular predetermined movement (e.g., an arcmovement) may be used which may signal to the device not to change theinformation displayed based on motion until the movement is completed.Once the predetermined movement is completed, the base referenceposition may be reset such that future movement of the device from thebase reference position further changes the information displayed. Thebase reference position may identify a baseline orientation of thedevice represented by baseline components of the motion data received bythe motion detection components of the device. In particularembodiments, gestures may be received, as determined by movement fromthe base reference position, to perform particular commands which changethe information displayed at the device.

As discussed with respect to various embodiments described above,handheld devices in accordance with particular embodiments may utilizemultiples types or modes of input to operate the device. Such inputmodes include motion input modes, such as a translation input mode and agesture input mode. While multiple input modes may sometimes be used incombination with each other, in some cases the handheld device may beset to recognize a certain mode type at one time. In some situations,the handheld device may be set to function based on multiple types ofnon-motion input and only one type of motion input (e.g., translation orgesture) at a particular time.

To facilitate this flexibility of the handheld device in recognizingmultiple input modes, in particular embodiments a certain trigger may beused to switch between input modes. For example, a user may press aparticular key or may move the device in a certain manner (e.g., aparticular gesture) to switch input modes. In some cases where anapplication of the device recognizes and functions based upon multipletypes of motion input, a particular key may be pressed or a particulargesture may be formed using the device to switch between a translationmotion input mode and a gesture motion input mode. The trigger may alsocomprise the mere switch from one application to another or the switchfrom one displayed image to another. In some situations, the trigger mayswitch between a non-motion input mode and a motion input mode. Anyparticular user-action may be implemented to act as a trigger to switchbetween different input modes, such as between different motion inputmodes. In some embodiments, a voice command or physical action upon thedevice (e.g., a device or screen tap) may be utilized to switch inputmodes.

In particular embodiments, a user action that reengages motionsensitivity of a device may also contain other information that mightotherwise affect device behavior. For example, if a user makes onemotion to reengage translation sensitivity, it may render the devicemore sensitive to motion than if the user makes a different motion toreengage motion sensitivity. A reengaging motion may comprise a gesturethat indicates the user's identity or context, thereby engaging avariety of operational settings, such as user preferences.

As indicated above, particular embodiments include the ability toreceive motion input to control various functions, tasks and operationsof a handheld device and may be used to alter information displayed atthe device in the process. In some cases, such motion input may be inthe form of gestures, as opposed to mere translation-based input.Gesture input may be used to navigate through a multidimensional menu orgrid in some applications. For example, as discussed above with respectto the scrubbing process, a display of a handheld device may be smallerthan the amount of information (e.g., menu options, map information)that can be presented on the display. This may lead to a menu structurethat is narrow and deep. In many cases, broad, shallow menu structuresmay be preferred over a narrow, deep menu structure because a user isnot required to remember as much information concerning wherefunctionalities are located.

FIG. 10A illustrates an example of menu navigation using gesture input,in accordance with a particular embodiment. In the illustratedembodiment, a handheld device is used to navigate through virtualdesktop 120. Virtual desktop 120 includes a menu tree with menucategories 122 for selection. Each menu category 122 may includerespective sub-categories for selection. In some embodiments, menucategories 122 may comprise categories of functions, whilesub-categories of each menu selection may include the actual functionsunder each such category. In other embodiments, menu categories maycomprise nouns (e.g., “folder,” “document,” “picture”), whilesub-categories comprise verbs (e.g., “move,” “paste,” “cut”). If thehandheld device comprised a cellular phone, menu categories 122 mayinclude “calls,” “phone book,” “messages,” “planner,” “sounds,” “setup”or other items. Each menu category 122 may include functions which maybe accessed once a menu category 122 is selected. While two menu levelsare illustrated in FIG. 10A, it should be understood that amultidimensional desktop or display of information for motion interfacenavigation may include any number of selections (e.g., menus) utilizingany number of levels.

In the illustrated example, menu category 122 e has been selected, andsub-categories 124 of menu category 122 e are displayed as available forselection. Boxes 126 and 128 represent information displayed at thehandheld device for the user. As illustrated, virtual desktop 120includes more information, or menus, which can be displayed at thedevice at one time. A user may move the device according to particulargestures to navigate across or through the virtual desktop. Gestures mayalso be used to navigate through different menu levels and to make menuselections. As an example, a user may move device 10 in the form of aclockwise circle 130 to navigate a predetermined amount to the rightacross virtual desktop 120 (e.g., moving from information of box 126 toinformation of box 128). A particular menu category 122 may be selectedby an away gesture 132, or a downward gesture (e.g., to select menucategory 122 e), and thus display sub-categories 124 for selection.Similarly, to move to the left across virtual desktop 120, a user maymove device 10 in the form of a counterclockwise circle 134. In somecases, navigation may be accomplished through four gestures: a forwardgesture, a backward gesture, a left gesture and a right gesture. In someembodiments, gestures comprising motion vectors in perpendiculardirections may be used for navigation.

In particular embodiments, gestures may be used that are mirror imagesof other used gestures to execute opposite functions from thoseaccomplished by the other gestures. For example, a motion toward a usermight zoom in while an opposite motion, a motion away from the user, mayzoom out. Using mirror image or reciprocal gestures mapped to oppositefunctions may make a motion user interface for a device easier to learnand to use.

In some cases, the menu item at the center of the display may behighlighted for selection, while in other cases a particular gesture mayindicate which menu selection of a plurality of displayed selections auser desires to select. It should be understood that the menus or otherinformation through which a user may navigate using gestures may bepresented at the handheld device in any number of ways. In someembodiments, only one level of information (i.e., one menu level) may bedisplayed at once, while sub-levels or higher levels are not displayeduntil they are available for selection.

FIG. 10B illustrates example gestures which may be utilized to performvarious functions, such as functions enabling a user to navigate througha virtual desktop. The illustrated example gestures include an “up”gesture 133 to navigate in an upward direction through the desktop, a“down” gesture 135 to navigate down, a “left” gesture 136 to navigateleft, a “right” gesture 137 to navigate right, an “in” gesture 138 tonavigate in a direction towards the user and an “out” gesture 139 tonavigate away from the user. It should be understood that these are mereexample gestures and commands for particular embodiments, and otherembodiments may include different gestures or similar gestures mapped todifferent commands for navigating through a desktop or performing otherfunctions with a handheld device.

FIG. 11 illustrates another example of map navigation using motioninput, in accordance with a particular embodiment of the presentinvention. FIG. 11 includes a virtual desktop 140 representing aninformation grid divided into sixteen portions, each referenced by arespective letter (A, B, C, . . . P). Portions of virtual desktop 140are identified using reference letters only for purposes of describingparticular embodiments, and portions of virtual desktops in accordancewith other embodiments may or may not be identified in a deviceapplication by a reference character or otherwise. Virtual desktop 140includes more information that can be displayed at a particular handhelddevice at one time. Virtual desktop 140 may represent any suitableinformation through which a user may desire to navigate using a handhelddevice, such as a street map. A user may desire to navigate acrossvirtual desktop 140 to display different portions of information on thehandheld device display and may also desire to zoom in (and out of)virtual desktop 140 (i.e., change the granularity of the informationdisplayed) to more clearly view certain portions of the information ofvirtual desktop 140.

In the illustrated example, box 142 represents information currentlydisplayed at handheld device 10. Box 142 includes portions A, B, E and Fof virtual desktop 140. In particular embodiments, if a user desires tochange the information of desktop 140 displayed at the device to, forexample, information of boxes C, D, G and H, then the user can usemotion input to move box 142 representing the display of the device tothe right the necessary amount (two portions to the right in theillustrated example). Such motion input may comprise translation input(moving the handheld device 10 to the right an applicable amount tochange the information displayed) or a gesture input (moving thehandheld device 10 according to a particular gesture mapped to thisfunction). As an example, one gesture may be mapped to moving thedisplay one portion to the right, while another gesture may be mapped tomoving the display two portions to the right. Thus, using translationinput or gesture input, the user may navigate across desktop 140.

The handheld device 10 may also allow the user to zoom in on certaininformation displayed for a clearer view of such information, forexample, through translation input or gesture input. As an example usinggesture input, if the information displayed at the device included fourof the sixteen portions (e.g., box 142 displaying portions A, D, E andF), then a user may use one of four gestures, each mapped to zoom in ona particular portion, to zoom in on one of the four displayed portions.If the user moved the handheld device according to the gesture mapped tozoom in on portion B, then the device may display informationrepresented by box 144 (portions B1, B2, B3, B4, B5, B6, B7, B8 and B9)collectively forming the information of portion B of virtual desktop 140in an enlarged view. Thus, the information of portion B may be morelargely and clearly displayed. When viewing the information of box 144on the device, the user may zoom out or zoom in again on a particularportion currently displayed using appropriately mapped gestures. If theuser moved the handheld device according to a gesture mapped as zoomingin on portion B2 (this could be the same gesture as that used to zoom inon portion B when information of box 142 was displayed), then the devicemay display information of box 146 (portions B2 a, B2 b, B2 c, B2 d, B2e, B2 f, B2 g, B2 h and B2 i). The user may also be able to navigateacross the virtual desktop when zoomed in on a particular portion. Forexample, when zoomed in on portion B (viewing information of box 144),the user may use translation or gesture input to move across the virtualdesktop to view enlarged views of portions other than portion B. As anexample, when viewing information of box 144 the user may make a gesturethat moves the information displayed to the right such that the entiredisplay shows only information of portion C of virtual desktop 140(i.e., zoomed in on portion C showing portions C1, C2, C3, C4, C5, C6,C7, C8 and C9). It should be understood that a user may navigate throughthe information of virtual desktop 140 (both navigating across andzooming in and out) in any suitable manner using motion input.

As indicated above, any suitable gestures may be used to both navigateacross a virtual desktop (or across a particular level) and to navigatebetween or through different levels or dimensions of a multidimensionaldesktop. Moreover, in some embodiments motion, such as gestures, may beused to navigate across the multidimensional desktop, and non-motionactions may be used to select or navigate between dimensions. Suchnon-motion actions may include the pressing of a key in a device input.Thus, a combination of motion and non-motion actions may be used formultidimensional virtual desktop or menu navigation in particularembodiments.

Particular embodiments may allow gesture-based navigation through anysuitable application, such as a multidimensional grid, menu, calendar orother hierarchical application. For example, in a calendar applicationcertain gestures may be used to navigate within one level, such asmonths, while other gestures may be used to navigate between levels,such as between years, months, days, hours and events. Moreover,different applications implemented in the handheld device that use suchgesture navigation may use different gestures. Thus, the particularnavigation gestures may change according to the particular applicationin use. In some embodiments, translation-based interfaces may be used tonavigation through multidimensional information of a virtual desktop, asopposed to merely using gesture-based movements. For example, movementalong x and y-axes may be used to travel within a level of a hierarchy,while movement along a z-axis may be used to travel between levels ofthe hierarchy.

Another example might involve using a telephone directory withinstitutions, letters of the alphabet, names, contact details (e.g.,office, cellular and home phone numbers, email) and action to initiatecontact all along different levels of a hierarchy. In this example, thehierarchy may contain information (nouns) and actions (verbs). One couldmap this example onto only two axes with, for example, a y-axis beingused to select within a level of hierarchy and an x-axis being used tomove between levels. A z-axis could be used to confirm the actions andhelp prevent inadvertent execution of actions.

In some cases, the number of levels traversed may depend on themagnitude of the motion, particular in translation-based navigation.Moving the device a small amount may navigate one level at a time, whilemoving the device a large amount may navigate multiple levels at a time.The greater the magnitude of motion, the more levels may be navigated atonce. As applied to gesture-based motion inputs, different gestures maybe used to navigate different numbers of levels in the hierarchy atonce. These gestures may be different magnitudes of the same motion orentirely different motions.

The use of motion interface navigation through a multidimensionaldesktop or information display increases may allow menus to be flattenedsince a user may more easily navigate across a particular menu ordimension of a virtual desktop too large to fit on a device's display.As a result of the menu flattening, the user may be required to memorizeless information thus increasing functionality and capability of thedevice for the user.

As discussed above, handheld devices in particular embodiments allow auser to navigate across a virtual desktop using motion input. In somecases, a user may utilize a cursor to navigate across informationdisplayed at the handheld device. For example, certain information maybe displayed at the device, and the user may utilize motion input tomove the cursor around the device display and to select particular itemsdisplayed to perform certain functions. In some cases, motion input maybe utilized to move a cursor, while a non-motion action (such aspressing a button) may be used to select an item currently indicated bythe cursor. It should be understood that both gesture and translationmotion input may be utilized in various embodiments of cursornavigation.

In particular embodiments, information displayed may be fixed withrespect to the device, and the cursor may remain fixed in space suchthat movement of the device is used to navigate the cursor across theinformation. FIG. 12A illustrates an example utilizing this form ofmotion input cursor navigation. Display 147 represents a display of ahandheld device. In order to describe this cursor navigation example,display has been divided into a grid to show information beingdisplayed. The grid includes portions A-P. Display 147 includes a cursor148 between portions C, D, G and H. As stated above, in this embodimentthe information displayed remains fixed with respect to the device whenthe device is moved, and the cursor remains fixed in space. However, thecursor's position with respect to the information displayed changesaccording to motion input. When the device is translated to the right,according to right movement 149, the cursor is translated according to amotion opposite to the translation of the device.

Display 150 represents a possible display after the device is movedaccording to right movement 149, with cursor 148 now between portions A,B, E and F. It should be understood that since this example involvestranslation-based input, the magnitude of the movement of the device(e.g., to the right in this example) may directly affect the magnitudeof the movement of the cursor with respect to the information displayed.Display 152 represents another display after the handheld device hasbeen moved according to up movement 151, with cursor 148 now betweenportions I, J, M and N. As evident, the cursor will have moved down withrespect to information displayed since it remains fixed in space.Display 154 represents another display after the handheld device hasbeen moved according to left movement 153, with cursor 148 now betweenportions K, L, O and P. As evidence, the cursor will have moved to theright with respect to information displayed. Thus, in this form ofcursor navigation, motion of the device changes the position of thecursor on the information. In this manner for example, the handhelddevice may be moved instead of using a stylus to point to certainportions of information displayed.

At any point in the cursor navigation process, a user may utilize anyform of input (e.g., gesture, key press, etc.) to select or otherwiseperform a function according to the information currently indicated atthe cursor. For example, with respect to display 152, a user may use aparticular gesture or press a button to zoom in, select or perform someother function based on information between portions I, J, M and Ncurrently indicated by cursor 148.

As described above with respect to FIG. 12A, particular embodiments maytranslate the cursor in a motion opposite from the motion of the deviceto move the cursor across the information displayed. In one example ofimplementation, input motion of the device may be divided into motionalong each of three axes, two of which are parallel to the devicedisplay (e.g., an x-axis and a y-axis). While motion of the device inthe x-axis and y-axis plane changes information displayed at the devicebased on such motion, the cursor may be moved at the same time accordingto a translation vector that is opposite to the sum of movement in thex-axis direction and the y-axis direction to substantially maintain theposition of the cursor in space. In some cases, when the cursor wouldmove past a display edge off of the display according to the translationvector, the vector may be reduced in order to keep the cursor within thedisplay. Such reduction may include reducing one or more components ofthe translation vector to maintain the cursor within a certain distancefrom the display edge.

It should be understood that the division of information displayed intoportions A-P is done only for illustrating and describing theembodiments described above, and information displayed at handhelddevices of particular embodiments may not include such a division orother type of reference information.

FIG. 12B illustrates another form of motion input cursor navigation, inaccordance with a particular embodiment. In this example, the cursorremains in a fixed position with respect to the display while motioninput is used to navigate across a virtual desktop larger than thedevice display. FIG. 12B includes virtual desktop 158 which comprisesinformation through which a user may navigate using motion input at ahandheld device, such as a street map. Virtual desktop 158 includes moreinformation that can be displayed at a particular handheld device at onetime. In order to describe this cursor navigation example, virtualdesktop 158 has been divided into a grid to differentiate betweeninformation presented at the desktop. The grid includes 6 rows (A-F) and7 columns (1-7). Portions of the grid may be identified herein for thisexample using their row letter and column number (e.g., portion B7 orD2). It should be understood that the division of virtual desktop 158into portions referenced by row and column number is done only forillustrating and describing the embodiments described above, and thevirtual desktops of particular embodiments may not include such adivision or other type of reference information.

Box 160 represents information of virtual desktop 158 currentlydisplayed at a handheld device. Display 161 represents the display of ahandheld device showing information of box 160. Display 161 alsoincludes a cursor 159 positioned at the intersection of portions B2, B3,C2 and C3. As indicated above, as a user utilizes motion input to movearound the virtual desktop (i.e., to change the information displayed atthe device), the cursor remains in a fixed position with respect to thedisplay. However, the cursor's position changes with respect to theinformation of the virtual desktop displayed at the handheld device. Forexample, a user may utilize motion input to change the informationdisplayed at the device to that represented at box 162. The informationdisplayed at the device changes (to portions B5, B6, C5 and C6); andcursor 159 will remain fixed at the device display (e.g., in this caseat the center of the display) such that its position changes withrespect to the information of virtual desktop 158, as illustrated atdisplay 163. If the user desired to use motion input to change theinformation displayed at the device to that represented at box 164, theinformation displayed at the device changes to portions E3, E4, F3 andF4, as illustrated at display 165. Cursor 159 is positioned betweenthese illustrated portions in the center of the display since itsposition relative to the display remains fixed in this embodiment.

Thus, according to the form of cursor navigation illustrated withrespect to FIG. 12B, the cursor will remain at a position fixed withrespect to the device display, while its position with respect to theinformation of a virtual desktop changes. As discussed above withrespect to the embodiment described and illustrated with respect to FIG.12A, at any point in the navigation process, a user may utilize any formof input (e.g., gesture, key press, etc.) to select or otherwise performa function according to the information currently indicated at thecursor. For example, with respect to display 163, a user may use aparticular gesture or press a button to zoom in, select or perform someother function based on information between portions B5, B6, C5 and C6currently indicated by cursor 159.

It should be understood that any particular input, such as a gesture orkey press, may be used to switch cursor navigation modes at the device.For example, a user may switch between the translation-controlled cursormode of FIG. 12A and the fixed cursor mode of FIG. 12B.

As discussed above, particular embodiments allow a user to move handhelddevice 10 according to a gesture to perform particular functions oroperations. In some cases however, a user may not move the deviceaccording to the particular gesture intended, and the device may, as aresult, not be able to recognize the movement as the intended gesture.In order to indicate that a particular movement of the device by a useris recognized as a particular gesture, handheld devices in someembodiments provide feedback to notify the user that the movement was infact recognized as a gesture.

This feedback may comprise an audio format, such as speech, a beep, atone or music, a visual format, such as an indication on the devicedisplay, a vibratory format or any other suitable feedback format. Audiofeedback may be provided through a user interface speaker or headphonejack of device 10, and vibratory feedback may be provided through a userinterface vibration generation module of device 10. Audio, visual andvibratory feedback may be varied to provide capability for multiplefeedback indicators. As an example, vibratory feedback may be varied induration, frequency and amplitude, singly or in different combinationsover time. The richness and complexity of the feedback may be expandedby using feedback of different types in combination with one another,such as by using vibratory feedback in combination with audio feedback.In some cases, the feedback may be gesture-specific, such that one ormore recognized gestures have their own respective feedback. Forexample, when a certain gesture is recognized the device may beep in aparticular tone or a particular number of times, while when one or moreother gestures are recognized the beep tones or number of beeps maychange. The use of audio feedback may be especially valuable forgestures that do not have an immediately visible on-screen manifestationor function, such as calling a certain number with a cellular phone.Different types of feedback may also be context or application specificin some embodiments. Different contexts might include device state, suchas what application is in focus or use, battery level, and availablememory, as well as states defined by the user, such as quiet or silentmode. For example, a handheld device may utilize vibratory feedback inresponse to gesture input while in silent mode when audio feedback wouldotherwise be used. This feedback process may also be utilized by ahandheld motion input device of a computer or other component.

Similar to the feedback relating to gesture recognition described above,handheld devices in particular embodiments may also provide feedback forthe user in the event that a particular user movement was not recognizedas a gesture when the device is in a gesture input mode. For example, ifa motion appears intended to be a gesture, but cannot be assigned to aparticular gesture that is known to the device, it may play a soundindicating failure. This notifies the user that the user must makeanother attempt at moving the device according to the intended gesturefor the device to perform the operation or function desired. Thefeedback notifying the user that a movement was not recognized may alsocomprise an audio, visual, vibratory or other suitable format and may bea different feedback than that communicated when a particular movementis recognized by the device as a particular gesture. In order todetermine whether a user's intent was to input a gesture, handhelddevice 10 may look at certain characteristics of the movement that implythat the motion was intended to be a gesture. Such characteristics mayinclude, for example, the amplitude of the motion, the time course ofthe above-threshold motion and the number and spacing of accelerations.If a particular gesture is unrecognized by the device, a system ofgestural feedback may be used to determine the gesture intended. Forexample, audio feedback may indicate possibilities determined by thehandheld device, and the user may utilize gestures to navigate anauditory menu to select the intended gesture.

In particular embodiments, a system of audio or vibratory feedback maybe used such that a user could operate handheld device 10 without havingto resort to viewing display 12. For example, handheld devices in someembodiments may provide audio, visual or vibratory feedback to a usernavigating a menu or other information of a virtual desktop. In effect,this device feedback combined with the motion input of a user could actas a type of “conversation” between the user and the device. Asindicated above, multiple types and complexities of feedback may beutilized. The feedback process could be particularly advantageous inenvironments where it may be inconvenient, unsafe or impractical to lookat the device display (e.g., while driving or while in a darkenvironment).

It should be understood that feedback, such as audio, visual andvibratory feedback, may also be used in some embodiments in connectionwith translation motion input. For example, a feedback indicator may begiven when a user reached a limit or edge of a virtual desktop usingtranslation input.

FIG. 13 is a flowchart 170 illustrating a process for utilizing feedbackin response to motion input, in accordance with a particular embodiment.At step 172 of the process raw motion data is received at handhelddevice 10. As described above, the raw motion data may be received byany combination of accelerometers, gyros, cameras, rangefinders or anyother suitable motion detection components. At step 174, the raw motiondata is processed to produce a motion detector output indicative of themotion of the device. Such processing may include various filteringtechniques and fusion of data from multiple detection components.

At step 176, the device state may be checked, because in someembodiments the feedback for a particular motion depends on the state ofthe device when the motion is received. As indicated above, exampledevice states may include the particular application in focus or use,battery level, available memory and a particular mode, such as a silentmode. At step 178, the motion detector output is analyzed with respectto the device state. At step 180, it is determined whether the motionindicated by the motion detector output is meaningful or otherwiserecognizable given the particular device state. For example, aparticular gesture may perform a certain function in one application(e.g., a calendar application) while the gesture serves no function inanother application. If the gesture is recognizable or meaningful givethe state of the handheld device, then at step 182 feedback is provided.As indicated above, in particular embodiments feedback may be in anaudio, visual or vibratory format. In some cases the feedback may merelybe an indication that the device recognizes the gesture given the stateof the device. In other cases, the feedback may be a further query foradditional input, for example if the user was utilizing a particularapplication of the device that provided for a series of inputs toperform one or more functions. At step 184, the device behaves accordingto the motion input and device state, and the process may return to step172 where additional raw motion data is received.

If it is determined at step 180 that the motion indicated by the motiondetector output is not meaningful or recognizable given the particulardevice state, then the process proceeds to step 186. At step 186, it isdetermined whether the motion is above a particular threshold. Thisdetermination may be made to determine whether particular motion inputwas, for example, intended to be a gesture. As indicated above,threshold characteristics for this determination may include theamplitude of the motion input, the time course of the motion input andthe number and spacing of accelerations of the motion. If it isdetermined that the motion input was not above a particular threshold,then the process may return to step 172 where additional raw motion datais received. If, however, the motion input was above a threshold suchthat a gesture may have been intended but was not recognized ormeaningful given the device state, feedback is provided at step 188. Thefeedback may include audio, visual and/or vibratory feedback and mayindicate that the gesture was not recognizable or meaningful. Inparticular embodiments, the feedback may also provide a query regardingthe intended gesture or may otherwise provide the user with a number ofpotentially intended gestures from which the user may select theparticular gesture intended by the motion. It should be understood thatparticular embodiments may not include some of the steps described(e.g., some embodiments may not include the threshold determination ofstep 186), while other embodiments may include additional steps or thesame steps in a different order. As suggested above, particularembodiments may utilize motion input feedback (e.g., including feedback“conversations”) in any of a number of applications and ways and thetype and complexity of feedback systems may vary greatly in differentembodiments.

As indicated above, handheld devices according to particular embodimentsmay receive gesture motion input to control any number of functions ofany number of applications running at the device. Some applications thatutilize gesture involve may comprise mobile commerce (mCommerce)applications in which a mobile device, such as handheld device 10, isused to conduct various transactions, such as commercial or consumerpurchases. Many mCommerce applications utilize some form ofauthentication to authenticate a user, such as personal identificationnumbers (PINs), credit card information and/or possession of the mobiledevice. However, many forms of authentication can be “leaked.” They canbe shared intentionally or inadvertently. Another form of authenticationis a user's written signature, which does not suffer from such leakingproblems since forgery is typically difficult to accomplish and may beeasy to detect. Particular embodiments may utilize motion input toreceive a user's signature as a form of authentication in mCommerce orother transactions through the handheld device.

A written signature may be considered a two dimensional record of agesture. When utilizing a handheld device with motion input, a user'ssignature may be in three dimensions and may thus comprise a “spatialsignature.” Moreover, when combined with other forms of input receivedat the device, a user's signature can take on any number of dimensions(e.g., four, five or even more dimensions). For example, athree-dimensional gesture “written” in space using the device anddetected at motion detector 22 may be combined with key-presses or otherinputs to increase the number of dimensions of the signature.

These spatial signatures can be tracked, recorded, and analyzed bymotion detectors 22 of handheld devices. They can be recorded withvarying degrees of precision with varying numbers of motion detectorcomponents to serve as an effective form of authentication. A user'sspatial signature may take comprise a three-dimensional form based onthe user's traditional two-dimensional written signature or may compriseany other suitable gesture which the user records at the handheld deviceas his or her signature.

In some embodiments, the process for recognizing a spatial signature mayinvolve pattern recognition and learning algorithms. The process mayanalyze relative timings of key accelerations associated with thesignature. These may correspond to starts and stops of motions, curvesin motions and other motion characteristics. In some cases, some hash ofa data set of a points of a signature motion may be stored, andsubsequent signatures may be compared against the hash for recognition.This may further verify if the signature was genuine by determiningwhether it was unique. For example, in particular embodiments, asignature may be detected (e.g., by a signature detection module ofdevice 10) by comparing a particular movement of the device with respectto an initial or reference position. Such comparison may be made bycomparing a sequence of accelerations of the movement with apredetermined sequence of accelerations of a stored spatial signature.This determination may be made regardless of the scale of the user'sinput motion signature.

In some embodiments, the device can detect whether motion of the devicematches the signature by determining whether positions of the device inmotion relative to an initial position match the spatial signature.

FIG. 14 illustrates an example system 200 utilizing spatial signaturesas authentication for mCommerce transactions. System 200 includeshandheld device 10, mCommerce application 202, authenticator 204 andcommunication network 206. mCommerce application 202 may comprise anysuitable application for transacting business with a handheld device ofa user. Such transactions may include consumer purchases, such asproducts or services of a business or other user from a website, onlinebill paying, account management or any other commercial transaction.Authenticator 204 authenticates, or verifies, a spatial signature inputby the user at handheld device 10 to complete an mCommerce transaction.Authenticator 204 may store one or more spatial signatures of one ormore users for authentication in an mCommerce transaction. In someembodiments, authenticator may be located within handheld device 10,within mCommerce application 202 or at any other suitable location.Communication network 206 is capable of transmitting information or databetween components of system 200 and may include one or more wide areanetworks (WANs), public switched telephone networks (PSTNs), local areanetworks (LANs), the Internet and/or global distributed networks such asintranets, extranets or other form of wireless or wireline communicationnetworks. Communication network 206 may include any suitable combinationof routers, hubs, switches, gateways or other hardware, software orembedded logic implementing any number of communication protocols thatallow for the exchange of information or data in system 200.

In operation, when a user utilizes handheld device 10 to conduct atransaction with mCommerce application 202, the user may utilize motioninput to communicate an authentication signature, for example by movingthe device according to the user's three-dimensional signature. As anexample, a user might use their cellular phone at a point-of purchase(e.g., store) instead of a credit card. Instead of signing a paperdocument that then needs to be shipped and processed, the user couldsimply move device 10 according to the user's spatial signature. Asindicated above, the user's signature may include more than threedimensions in some embodiments. The signature may have been previouslyrecorded by the user using handheld device 10 or another mobile device,and the recorded signature may be stored at handheld device 10,mCommerce application 202, authenticator 204 or other suitable locationsuch as a signature storage database for signatures of multiplemCommerce users.

The motion of handheld device 10 may be processed at the device, andmotion output indicative of the motion may be transmitted to mCommerceapplication 202. mCommerce application 202 may communicate the motionoutput to authenticator 204 for verification that the motion inputreceived at device 10 was in fact the signature of the user attemptingto transact mCommerce. If authenticator 204 verifies the user'ssignature, then mCommerce application may complete the transaction withthe user. As indicated above, authenticator 204 may be located withinhandheld device 10 or at mCommerce application 202 in particularembodiments and may access signatures for verification stored at device10, mCommerce application 202 or any other suitable location.

Authentication may also be used by handheld devices in a non-mCommerceapplication, for example when electronic security is desired to performfunctions such as sending private or secure data using the device. Auser desiring to transmit data or other information using handhelddevice 10 may use their spatial signature in the encryption process.Spatial signatures may be used in any of a variety of ways to securedata for communication through a network and may be utilized inconnection with public/private key encryption systems. For example, insome embodiments handheld device 10 may authenticate a user's signaturereceived through motion input and then use its own private key toencrypt data for transmission. In other cases, data may be communicatedto handheld device 10 such that an intended recipient must input theirspatial signature to receive the decrypted data. In some embodiments,data may be communicated to a computer wirelessly connected to handhelddevice 10, and the intended recipient must use handheld device 10 as away to communicate the user's signature to the computer for datadecryption. Moreover, in particular embodiments, a user's spatialsignature itself may represent an encryption key such that motion of thedevice generates the encryption key instead of the signature motionmerely being used for authentication. In some cases a device mayrecognize a combination of accelerations as a signature by convertingthe signature into the equivalent of a private key. The handheld devicemay then use the private key as part of an authentication process for atransaction.

In particular embodiments, a spatial signature may be used to managephysical access to a building or event. For example, the signature inputby a user at a device may be checked against a list of people allowed toenter as IDs are checked at “will call” for an event.

In particular embodiments, a user may utilize motion input for handhelddevice 10 to control other devices, such as audio/video equipment, homeappliances and devices, computing devices or any other device capable ofbeing controlled by a handheld device. Devices may be controlled byhandheld device 10 through communications interface 20 of device 10utilizing any of a number of wireless or wireline protocols, includingcellular, Bluetooth and 802.11 protocols. In some embodiments, device 10may receive motion input to control, through wireless or wirelinecommunication, other devices through a network. Thus, devices controlledthrough motion input of device 10 may be at any location with respect todevice 10, such as in the same room or across a country. Moreover,control of the other device may be implemented through any number ofintermediate devices (e.g., through a network).

As an example, if handheld 10 device were a Bluetooth-enabled cellularphone, then particular gestures or other motion of the cellular phonemay wirelessly communicate commands to control another device, such as alaptop across a room to drive a PowerPoint presentation. Other deviceswhich may be controlled through motion input of handheld device 10 mayinclude televisions, radios, stereo equipment, satellite receivers,cable boxes, DVD players, digital video recorders, lights, airconditioners, heaters, thermostats, security systems, kitchen appliances(e.g., ovens, refrigerators, freezers, microwaves, coffee makers, breadmakers, toasters), PDAs, desktop and laptop PCs, computer peripheralequipment, projectors, radio controlled cars, boats and planes and anyother device. As another example, a commuter may shake their cellularphone in a certain manner to tell their heater at home to turn on beforethe commuter arrives at home. In some embodiments, a handheld device mayreceive and process raw motion data to determine commands or intendedfunctions for communication to other devices. In other embodiments, amotion detector of a handheld device may output raw data received fromits motion detection components for communication to one or more devicescontrolled by device 10 through the motion of device 10. As a result,different devices controlled by device 10 may treat the same raw motiondata of device 10 differently. For example, a particular gesture ofdevice 10 may perform different functions of different devicescontrolled by device 10.

FIG. 15 illustrates an example system 220 in which handheld device 10controls multiple other devices through motion input of device 10.System 220 includes handheld device 10, laptop 222 and remote device 224connected, through wireless or wireline links, to handheld device 10through communication network 226. Handheld device 10 receives rawmotion data of a particular motion of the device through motiondetection components, such as accelerometers, cameras, rangefindersand/or gyros. The raw motion data is processed at the handheld device.Particular databases, such as gesture and gesture mapping databases, maybe accessed to determine a matching gesture and intended function basedon motion tracked by a control module of the device. The intendedfunction may be for another device to be controlled by handheld device10, such as laptop 222 or remote device 224. Thus, the motion input isthe interface for the underlying operational signal communicated fromdevice 10 to the controlled device. In other embodiments, the raw motiondata or other data merely indicating a particular motion input fordevice 10 may be directly sent to laptop 222 and/or remote device 224without determining a function at device 10. In these embodiments,laptop 222 and/or remote device 224 may themselves process the rawmotion data received from handheld device 10 to determine one or moreintended functions or operations they should perform based on the rawmotion data. In some embodiments, a user of device 10 may indicate todevice 10 through motion or otherwise the other devices that handhelddevice 10 should communicate raw motion data or intended functions ofsuch other devices, as applicable. While two devices controlled byhandheld device 10 are illustrated, it should be understood thatparticular embodiments may include any number of devices of varyingtypes to be controlled by handheld device 10 through motion input asdiscussed above.

As indicated above, particular embodiments include the ability tocontrol other devices, such as other local or remote devices, throughmotion input of handheld device 10. In some embodiments a user ofhandheld device 10 selects the other device that a particular motioninput of device 10 is intended to control. For example, a user may useinput 14 of handheld device 10 (e.g., by pressing a button or moving atrackwheel) to select a local or remote device to control before movingdevice 10 according to a particular motion mapped to a function oroperation desired for the other device. In particular embodimentshowever, a user may move handheld device 10 according to a particulargesture in order to select the other device (e.g., other local or remotedevice) to be controlled at the time through motion input of device 10.Thus, particular embodiments provide gesture motion selection of otherdevices to be controlled by handheld device 10.

Handheld device 10 may include a device selection module operable todetect a device selection gesture which indicates that a user desires tocontrol a particular device. Each controllable device may include itsown gesture command maps which correlate gestures to be input usingdevice 10 and commands of the controllable device. A control module ofthe handheld device may select a particular command map corresponding tothe controllable device selected for control. In some embodiments,device 10 may include a device locator operable to detect, for each of aplurality of remote devices, a direction from the handheld device toeach remote device. In this case, the user may move handheld device 10in the direction of a particular remote device the user desires tocontrol in order to select that remote device for control.

While motion input for device 10 may be used for such control of theother devices, other types of input (e.g., utilizing input 14) may alsobe used to control other local or remote devices selected for control bygesture input. In some embodiments different gestures may be each mappedto control a different device. In other embodiments, device 10 maydisplay possible other devices for control and particular gesture(s) toutilize to indicate a user's selection as to which other device the userdesires to presently control through device 10. Handheld devicesaccording to the present invention may utilize any particular manner ofgesture selection of one or more local or remote devices to becontrolled by the handheld devices.

As discussed above, particular embodiments include handheld devices 10capable of detecting motion of the device through a motion detector 22to modify the behavior of the device in some way according to the motiondetected. Handheld devices 10 in some embodiments are capable ofmodeling their particular environment and subsequently modifying theirbehavior based on such environment. One distinction between modeling anenvironment of a handheld device and detecting a particular motion ofthe device is that in the former case there may be reasoning involvedand in the latter case there may be no such reasoning. As an example, ifa handheld device changes its behavior when moved according to aparticular gesture, that may be considered sensing or detecting aparticular motion and reacting based on the motion detected. If, on theother hand, the handheld device determines that it is sitting face downon a table and reacts accordingly, that may be considered modeling itsenvironment. As another example, if a handheld device moves to the leftand changes its behavior based on such movement, that may be considereddetecting motion and reacting. If the handheld device finds itself infree fall and powers-down so as to survive an impending collision withthe ground, that may be considered modeling its environment. A furtherdistinction may be that environmental modeling may not require animmediate response to a user input, while detecting an event, such as aparticular motion, generally does require such an immediate response.Modeling an environment may thus involve sensing or detecting a patternof motion (or lack thereof), matching it to a predefined set ofenvironmental conditions and modifying the behavior of the device basedon the modeled environment. The behavior implemented based on theenvironment modeled may also change based on a particular application inuse or in focus. In some cases, the device may change its sensitivity toparticular motions based on the environment modeled.

As an example, a handheld device may recognize, through accelerometersor other motion detection components, that it is at rest on anapproximately horizontal surface. Such recognition may result from adetermination that the device is not moving, or still, with a static 1 gof acceleration orthogonal to a surface. The device may be able todifferentiate resting on a table from resting in a user's hand, forexample, because a user's hand typically will not be able to hold thedevice perfectly still. The device may, as a result, behave in a certainmanner according to the recognition that it is at rest on anapproximately horizontal surface. For example, if handheld device 10recognized that it was lying at rest on a table, it may power off afterlying in such position for a certain amount of time. As another example,a cellular phone in a vibrate mode may vibrate more gently if itrecognizes it is on a table upon receipt of a call or upon any otherevent that may trigger vibration of the phone. In some embodiments, thedevice may recognize its orientation while lying on a table such that itmay behave in one manner when lying in a “face down” position (e.g., itmay power off) while it may behave in a different manner when lying in anon-face down position. If handheld device 10 comprised a cellularphone, it may enter a speaker mode when it is on a call and recognizesthat it is placed by a user in a “face up” position on a table while onthe call. If, on the other hand, the cellular phone is on a call and isplaced face down on the table, it may enter a mute mode.

As another example, handheld device 10 may recognize through a briefperiod of approximately 0 g that it is in free-fall and then may behaveto reduce damage due to impending impact with the ground or anothersurface. Such behavior may include, for example, powering down chipsand/or hard drives, retracting lenses, applying covers or any otherdevice behavior. In particular embodiments, non-hand-held devices ordevices that do not otherwise detect motion for input may also be ableto model their environment and to behave based on the environmentmodeled. As an additional example, acceleration patterns may be detectedto recognize that a handheld device 10 is in a moving environment (e.g.,being held by a user in a car or on a train) and may adjust varioussensitivities, threshold and/or other characteristics to enable betterperformance of the device in that environment.

In other embodiments, handheld device 10 may comprise a digital camera.Through its motion detection components, the camera may determinewhether it is on a tripod or is being held by a user when a picture istaken. The camera may set a shutter speed for the picture based on thatdetermination (e.g., slow shutter speed if on a tripod or fast shutterspeed if being held by a user).

If handheld device 10 comprised a device that utilized a cradle forsyncing up with another device, such as a PC, then device 10 mayrecognize that it is in the cradle based on its stillness (or supportedstate) and its particular orientation. The device may then operate orfunction according to its state of being in the cradle (e.g., it maythen sync up with its associated PC).

FIG. 16 is a flowchart 230 illustrating an environmental modelingprocess, in accordance with a particular embodiment. At step 232, rawmotion data is received at handheld device 10. As described above, theraw motion data may be received by any combination of accelerometers,gyros, cameras, rangefinders or any other suitable motion detectioncomponents. At step 234, the raw motion data is processed to produce amotion detector output from which motion and orientation of the deviceis determined at step 236. Boxes 237 represent example motions andorientations of the device, such as rotating around the z-axis in box237 a, translating along the x-axis in box 237 b, oriented at particularangles α, θ, ω in box 237 c and still in box 237 n. These are mereexample motions and orientations of the device, and any number ofmotions may be utilized that are determined at step 236. In someembodiments, the determined orientations may comprise an orientation ofthe device with respect to gravity.

At step 238, handheld device 10 determines its environment based on themotion and orientation determined at step 236. Boxes 239 representexample environments of the device, such as face down on a table in box239 a, falling in box 239 b, on a train in box 239 c and held in hand atbox 239 n. Any number of environments may be determined based on motionsand orientations determined at step 236. In particular embodiments, theenvironmental determination may also be based on a history of thedevice, such as a motion/orientation history. For example, whenimplementing a speaker mode function of a cellular phone, the device maydetect a quite period when horizontal in the middle of a call after ashort jarring is detected (e.g., the short jarring caused by a userplacing the phone face up on a table). The phone can detect that it wasjarred so that stillness and a perpendicular position relative togravity may take on a different meaning than had the jarring notoccurred. Thus, the determination of the environment may be based on themotion and orientation of the device and its history. The history maycomprise a previous motion/orientation of the device or any otherinformation relating to a device's history.

At step 240, the determined environment is mapped to a particularbehavior. The mapped behavior may depend on any number of factors inaddition to the determined environment, such as desired characteristicsof the particular user using the device at the time or the particularapplication in use or focus at the time. For example, the behavioraccording to a particular modeled environment may include engaging amute function of the handheld device in box 241 a, powering down chipsof the device to survive an impact in box 241 b and increasing a motionactivation threshold of the device in box 241 n. The mute behaviorindicated in box 241 a may be implemented when a cell phone'senvironment comprises laying face down on a table while on a call. Thepowering down chips behavior in box 241 b may be implemented when theenvironment of handheld device 10 comprises a free fall of the device.The increasing a motion activation threshold behavior of box 241 n maybe implemented when a handheld device's environment comprises being in acar or train where bumpiness may require a greater movement thresholdfor a user's motion input to register as an intended input. Particularembodiments may include any number of behaviors mapped to one or moremodeled environments. At step 242, the handheld device behaves accordingto the behavior to which its environment is mapped at step 240.

As indicated above, users may move handheld devices according toparticular gestures to cause the devices to perform desired functions,operations or tasks. In particular embodiments, gestures used as motioninput for the device may comprise pre-existing symbols, such as lettersof the alphabet, picture symbols or any other alphanumeric character orpictographic symbol or representation. For example, gestures used asmotion input may mimic upper and lower case members of an alphabet inany language, Arabic and Roman numerals and shorthand symbols.Preexisting gestures may be used for handheld input devices for otherlocal and remote devices as well. Using preexisting gestures forhandheld device input may facilitate the learning process for users withrespect to gesture motion interfaces.

FIG. 17 illustrates example gestures which may be mapped to particularfunctions. For example, if handheld device 10 comprised a cellularphone, a user may move device 10 in the form of heart 250 to call theuser's girlfriend, boyfriend or spouse or house 252 to call the user'shome. As another example, if handheld device 10 were a PDA or otherdevice running an application managing files or data, moving the devicein the form of C-gesture 254 may be a command for copying data,O-gesture 256 may be a command for opening a file, D-gesture 258 may bea command for deleting data and X-gesture 260 may be an exit command fora file or application. The logical connection between gestures and theirintended functions or operations (e.g., “O” for opening a file) furtherfacilitates user interaction and learning.

Any number of pre-existing symbols may be used as gestures for motioninput as commands for performing any number of functions, operations ortasks of a handheld device. Many preexisting gestures typically exist intwo dimensions. Handheld device 10 may recognize such gestures. In somecases, for example, handheld device 10 may disable receipt of aparticular dimension so that any movement in a third dimension when auser is attempting to input a two-dimensional gesture is not received ordetected in order to facilitate recognition of the two-dimensionalgesture. In some embodiments, handheld device 10 may receivethree-dimensional gestures that may be based on preexistingtwo-dimensional gestures. Receiving and detecting three-dimensionalgestures increases the capabilities of the device by, for example,increasing the number and types of gestures which may be used as motioninput.

FIG. 18 is a flowchart 270 illustrating the utilization of a preexistingsymbol gesture, the letter “O,” as motion input. As illustrated in step272, a user moves handheld device 10 in the form of the letter “O.” Atstep 274, handheld device 10 receives raw motion data of the “O”movement from motion detection components and process such raw motiondata at step 276 to determine the actual motion of the device. At step278, handheld device 10 accesses a gesture database 280 which mayinclude a plurality of gestures recognizable by the device to map themotion to the gesture “O.” The plurality of gestures of the gesturedatabase may each be defined by a series of accelerations of a movement.The actual motion of the device may be matched to a series ofaccelerations of one of the gestures of the gesture database. At step282, handheld device 10 maps the gesture “O” to a particular function byaccessing a function database 284 (or a gesture mapping database) whichmay include a plurality of functions that may be performed by one ormore applications running on the device. In particular embodiments, thegesture and function databases may be comprised in memory 18 of thedevice. The particular function mapped to the gesture “O” may depend ona particular application in focus or being used by the user at the time.For example, in some applications “O” be comprise a command to open afile, while in other applications it may comprise a command to call acertain number. In some cases, one gesture may be mapped to the samefunction for all applications of the device. At step 286, the devicebehaves according to the mapped function, such as opening a file.

As indicated above, gestures used as motion input for handheld device 10may have different meanings (e.g., functions, operations, tasks) basedon a particular context, such as a particular application in use or infocus, a particular device state with respect to an application orotherwise, a particular modeled environment or any combination of theseor any other context. For example, a particular gesture may be mapped asa command to scroll a page up when running a web browser at the device,while the gesture may be mapped as a command to examine a different datewhen running a calendar program. The ability for particular gestures tobe mapped to different commands depending on the context, such as theapplication in use, increases the functionality of the device.

Handheld devices in some embodiments may be able to utilize simplermotion detection components if gestures are mapped to different commandsdepending on the context. As an example, a handheld device may includeparticular motion detection components such that the handheld device mayonly be able to recognize and distinguish between twenty differentgestures. If each gesture is mapped to a different function for each offour different applications, then the ability to only recognize twentyunique gestures still provides eighty different functions on the device(twenty for each application). If each gesture were mapped to its ownfunction, no matter what application was in focus; then the overallcapability of the device would be reduced, and some gestures wouldlikely not be used in some applications. The ability to use less complexcomponents that are able to recognize and distinguish between fewergestures as a result of mapping gestures to a plurality of functionsdepending on the context can lead to reduced costs in the componentsutilized in the device and can also simplify the task of physicallylearning gestures required to control the device. As indicated above,gestures may be mapped to different functions, operations or tasksdepending on the application in use, device state, modeled environmentor other context. In some cases, gestures may be mapped to differentfunctions depending on the state of a particular application. Forexample, in the case of a word processing program, some gestures mayhave some functions when in one state of the program (e.g., a menustate) while the same gestures may have different functions when inanother state of the word processing program (e.g., a document editingstate). In this case, a command map associated with the gesture functionmappings may include gesture mappings for each such state.

FIG. 19 is a flowchart 290 illustrating the use of context-based gesturemapping, in accordance with a particular embodiments. In the illustratedembodiment, a gesture has different functions assigned based on theapplication in focus. At step 292, handheld device 10 receives rawmotion data of a particular gesture movement and process such raw motiondata at step 294 to determine the actual motion of the device. At step296, handheld device 10 maps the motion to a gesture, for example, byaccessing a gesture database. At step 298, handheld device 10 determineswhich application is in focus. For example, if the device were capableof running four different applications, then it would determine which ofthe four was in focus or was being used at the time. The device thenperforms the function mapped to the gesture according to the applicationin focus. The identification of such function may be accomplished insome embodiments by accessing a function database which may also bereferred to as a gesture mapping database since it correlates gesturesof a gesture database to functions. In the illustrated embodiment, ifApplication 1 is in focus, then the device performs Function 1 at step300 a; if Application 2 is in focus, then the device performs Function 2at step 300 b; if Application 3 is in focus, then the device performsFunction 3 at step 300 c; and if Application 4 is in focus, then thedevice performs Function 4 at step 300 d.

As a further example of context-based gesture mapping, a handheld devicewith phone and PDA capabilities may run four applications: a phoneapplication, a calendar application, a file management application andan e-mail application. A gesture input mimicking the letter “S” may havedifferent functions depending on the application in focus. For example,if the phone application is in focus, then receiving the gesture input“S” may be a command for calling a particular number designated to the“S” gesture. If the calendar application is in focus, then receiving thegesture input “S” may be a command for scrolling to the month ofSeptember in the calendar. If the file management application is infocus, then receiving the gesture input “S” may be a command for savinga file. If the e-mail application is in focus, then receiving thegesture input “S” may be a command for sending an e-mail. Particularembodiments contemplate great flexibility in the ability to map gesturesto different functions depending on the context.

As discussed above, gestures may have different functions depending on aparticular context at the time. In particular embodiments, handhelddevices may be customizable to allow users to assign device functions topre-defined gestures. The functions may be context-based such that somegestures may have different functions depending on an application inuse, a device state or a modeled environment. Handheld devices in someembodiments may allow different users of the same device to assigndifferent functions to the same gesture, and such functions may also becontext-based as discussed above.

For example, a handheld device 10 may be utilized by a number ofdifferent users at different times. Each user may assign differentfunctions for the same gestures. When the handheld device receives agesture input, it must thus know which user is using the device at thetime to determine which function the user intends the device to perform.The device may determine the user in any of a variety of ways. In someembodiments, users may log into the device prior to use by using ausername and password or otherwise. In other embodiments, the handhelddevice may be able to identify the user based on the manner in which theuser moves the device for motion input, such as the way the user forms agesture using the device. As indicated above, each user may also assigncommands to gestures based on context, such as based on the applicationin focus at the device. The ability for the handheld device to mapfunctions to gestures based on particular users further increases thedevice's capabilities and flexibility, particularly if the device isable to recognize and distinguish only a particular number of gestures.

FIG. 20 is a flowchart 310 illustrating the use of user-based gesturemapping, in accordance with a particular embodiment. In the illustratedembodiment, a gesture has different functions assigned based on the userusing the device. At step 312, handheld device 10 receives raw motiondata of a particular gesture movement and process such raw motion dataat step 314 to determine the actual motion of the device. At step 316,handheld device 10 maps the motion to a gesture, for example, byaccessing a gesture database. At step 318, handheld device 10 determineswhich user is using the device. Such determination may be made, forexample, through a log in system in which users log into the deviceprior to use. Handheld device 10 may determine the current user throughother suitable methods as well. At steps 320, the device performs thefunctions assigned to the gesture input based on the user using thedevice. In the illustrated embodiment describing the process with fourpossible users, if User 1 is using the device, then the device performsFunction 1 at step 320 a; if User 2 is using the device, then the deviceperforms Function 2 at step 320 b; if User 3 is using the device, thenthe device performs Function 3 at step 320 c; and if User 4 is using thedevice, then the device performs Function 4 at step 320 d.

As indicated above, in some embodiments, gestures may be assigneddifferent functions based on both users using the device and a context.In this situation, the illustrated flowchart 310 described above mayhave an additional step to determine the context at the time (e.g., step298 of flowchart 290 determining the application in focus). Theparticular function desired for performance by a certain gesture thusdepends on both the user using the device at the time and the context,such as the particular application in focus at the time.

As previously discussed, some embodiments include handheld devices withthe ability to receive preexisting symbols as gestures for motion input.Some of those embodiments as well as other embodiments may include theability for user's to create their own gestures for mapping to functionsand/or keys. The gestures may comprise any user-created symbol or othermotion that the user desires to utilize as motion input for one or moreparticular functions, operations or tasks that the device is able toperform. Users may be able to create motions with some personalsignificance so they may more easily remember the motion's command orintended function.

FIG. 21 is a flowchart 330 illustrating the assignment process foruser-created gestures, in accordance with a particular embodiments. Atstep 332, an indication is received from a user for gesture creation.The indication may be received in any of a variety of ways using anysuitable input format (e.g., keys, trackwheel, motion, etc.). The usermay move the device according to the user-created gesture such that rawmotion data for the user-created gesture is received at the handhelddevice at step 334. The raw motion data may comprise a sequence ofaccelerations of a movement after stabilization of the device from abase reference position until an indication is received to stoprecording the reference positions. Indications to start and stoprecording a user-created gesture may include motion or non-motionindications (e.g., key presses and key releases). The raw motion data isprocessed at step 336. At step 338, the motion is stored as a gesture,for example, at a gesture database. In particular embodiments, theindication for gesture creation may be received after the user moves thedevice according to the user-created gesture. For example, the user maymove the device according to a user-created gesture that is currentlyunrecognizable by the device. The device may query the user to determineif the user desires to store the unrecognized gesture for a particularfunction. The user may respond in the affirmative so that the user mayutilize the gesture as motion input in the future.

At step 340, function mapping information for the gesture is receivedfrom the user. The function mapping information may comprise functions,operations or tasks of the device that the user desires the user-createdgesture to command. In particular embodiments, such function mappinginformation may comprise a series of functions (e.g., a macro) that onegesture may command. The user may assign different functions for agesture according to an application in focus. In some cases, a user maydesire to map different gestures to different keys or keystrokes of thedevice. One examples of mapping a series of functions to a gesture mayinclude mapping a long string of characters to a gesture (e.g.,telephone numbers including pauses, where appropriate). At step 342, thefunction mapping information is stored, for example, at a functiondatabase or gesture mapping database.

As indicated above, it may be difficult for a user to move handhelddevice 10 in the same precise manner for one or more gestures each timethose gestures are used as input. Particular embodiments thus allow forvarying levels of precision in gesture input. Precision describes howaccurately a gesture must be executed in order to constitute a match toa gesture recognized by the device, such as a gesture included in agesture database accessed by the device. The closer a user generatedmotion must match a gesture in a gesture database, the harder it will beto successfully execute such gesture motion. As discussed above, inparticular embodiments movements may be matched to gestures of a gesturedatabase by matching a detected series of accelerations of the movementsto those of the gestures of the gesture database.

As the precision of gestures required for recognition increases, one mayhave more gestures (at the same level of complexity) that can bedistinctly recognized. As an example, if the precision required waszero, then the device could only recognize a single gesture but it wouldrecognize it easily because anything the user did would be recognized asthat gesture. If, however, the precision required was infinite then itwould be virtually impossible for a user to form a gesture that wasrecognized by the device, but the device could support an infinitenumber of gestures with only infinitesimal differences between them. Onearea in which the precision requirement is especially applicable is inthe area of spatial signatures. With spatial signatures, the level ofprecision correlates well with the level of security.

In particular embodiments, the precision required by handheld device 10for gesture input may be varied. Different levels of precision may berequired for different users, different regions of the “gesture space”(e.g., similar gestures may need more precise execution for recognitionwhile gestures that are very unique may not need as much precision inexecution), different individual gestures, such as signatures, anddifferent functions mapped to certain gestures (e.g., more criticalfunctions may require greater precision for their respective gestureinputs to be recognized). Moreover, in some embodiments users may beable to set the level(s) of precision required for some or all gesturesor gestures of one or more gesture spaces. As an example, a user may setthe precision required for spatial signatures higher than for othergestures for the user thus increasing security for spatial signatureinput.

As indicated above, in particular embodiments gestures may be recognizedby detecting a series of accelerations of the device as the device ismoved along a path by a user according to an intended gesture.Recognition occurs when the series of accelerations is matched by thedevice to a gesture of a gesture database.

In some embodiments, each gesture recognizable by handheld device 10, oreach gesture of a gesture database, includes a matrix ofthree-dimensional points. In addition, a user movement intended as agesture input includes a matrix of three-dimensional points. Handhelddevice 10 may compare the matrix of the movement with the matrices ofeach recognizable gesture (or each gesture in the gesture database) todetermine the intended gesture. If a user moves the device such that themovement's matrix correlates to each point of an intended gesture'smatrix, then the user may be deemed to have input the intended gesturewith perfect precision. As the precision required for gesture input isreduced, then the greater the allowable differences between a usergesture movement and an intended gesture of a gesture database forgesture recognition.

FIG. 22 illustrates three gestures input using a handheld device withvarying levels of precision. In the illustrated embodiment, the intendedgesture comprises an “O.” Gesture movement 350 is input as a perfect“O,” or with 100% precision for the intended gesture. Gesture movement352 is input with less than 100% precision as it does not form a perfect“O.” Gesture movement 354 is input with less precision than gesturemovement 352. The precision requirement for the input of gesture “O” maybe set at handheld device to accept varying levels of precision. Forexample, the precision level may be set such that only gesture movement350 is recognized as gesture “O,” gesture movements 350 and 352 are bothrecognized as gesture “O” or gesture movements 350, 352 and 354 are allrecognized as gesture “O.” As indicated above, the higher the precisionrequirement, then the more space available for additional recognizablegestures. For example, if the precision level for handheld device 10were set such that only gesture movement 350 was recognized as gesture“O,” then gesture movements 352 and 354 may be recognized as other,distinct gestures.

In particular embodiments, handheld devices may alter gesturesrecognized for performing particular functions based on a user'spersonal precision. In this manner, a handheld device may have dynamiclearning capability of gesture mappings. For example, if a particulargesture of a gesture database is mapped to a particular function, and auser's repeated attempts to input the gesture lack precision in aconsistent manner; then the handheld device may alter the gesture in thegesture database to match the consistent gesture movement of the usersuch that the user's consistent gesture motion input will be mapped tothe particular function.

As an example, if a particular gesture comprises a square motion and auser's motion intended for the gesture comprises more of a triangularmotion on a consistent basis (e.g., on multiple consecutive times), thenthe handheld device may be able to recognize this consistent differencein an intended gesture and the actual user motion to change the intendedgesture (e.g., a square) in the gesture database mapped to the functionintended to the user's actual consistent motion (e.g., triangle). Thus,after such change is made, any time the user inputs the triangle gesturethen the function previously mapped to the square gesture will becommanded. The device may determine the gesture intended in any of avariety of ways, such as through two-way communication with the userthrough any form of input. In particular embodiments, this dynamiclearning of users' input characteristics may be applied on auser-specific basis. For example, in the example described above anotheruser may still input the square gesture using the same handheld deviceto command the same function.

As indicated above, as the precision of user motion with respect tointended gestures increases, then the number of gestures available formapping to functions increases. In some embodiments, handheld devicesmay recognize that a user's precision increases over time and thedevices may, as a result, increase the gestures available for use.Increasing gestures available for input may also increase the functionscapable of being commanded through gesture input.

As an example, a user's personal precision for inputting gestures may besuch that the user is only able to input a certain number of gesturesthat will be recognized by the handheld device. However, over time, theuser's personal precision may increase. This increase may be recognizedby the handheld device and, as a result, the device may enableadditional gestures that the user may use as gesture input. In someembodiments, the enabling of additional gestures may occur when theuser's precision increases over a particular precision threshold, or acertain precision level. Since the user's precision has increased, thehandheld device will be able to recognize when the user attempts toinput these additional gestures. As indicated above, providingadditional gestures for input by a user may also increase the number offunctions that the user is able to command through gesture input, sinceeach gesture may be mapped to command a different function.

Handheld devices in particular embodiments may also allow users to setand vary noise thresholds of the device. Noise thresholds are themagnitude of motion of the device that must be detected in order to beconsidered intended motion input (e.g., an intended gesture) of theuser. For example, if noise thresholds are set low, then minimal motionof the device may be considered by the device as motion input. However,if noise thresholds are set high, then greater movement of the devicewould be required before the motion is considered intended input fromthe user. If, for example, a user is travelling in a car on a bumpyroad, the user may desired to set the noise threshold higher so thatwhen the device moves as a result of bumps in the road then suchmovement may not be considered by the device to be intended motioninput.

In particular embodiments, noise thresholds may automatically change atthe device based on a modeled environment. For example, if a devicedetermines that the environment comprises traveling in a car, then thedevice may automatically increase the noise threshold so that minimalmovements resulting from the car will not register as user-intendedmotion.

FIG. 23 is a flowchart 370 illustrating a gesture recognition processutilizing a number of features described herein, in accordance with aparticular embodiment. At step 372, raw motion data of a particulargesture movement is received. The raw motion data is processed at step374 where the actual motion of the device is determined. Such processingmay include various filtering techniques and fusion of data frommultiple detection or sensing components. At step 376, the actual motionis mapped to a gesture. Mapping the actual motion to a gesture mayinclude accessing a user settings database 378, which may include userdata 379 comprising, for example, user precision and noisecharacteristics or thresholds, user-created gestures and any otheruser-specific data or information including user identities 381.User-specific information may be important, for example, becausedifferent users of the handheld device may have different settings andmotion input characteristics. For example, an older user may have lessprecision than a younger user when inputting gestures such that theolder person may have fewer gestures available. Moreover, a moreexperienced user may have more device functionality available throughgesture input.

User settings database 378 may also include environmental modelinformation 380 which may factor in determining the gesture applicableat the time. As discussed above, through environmental modeling, thedevice can internally represent its environment and the effect thatenvironment is likely to have on gesture recognition. For example, ifthe user is on a train, then the device may automatically raise thenoise threshold level. The device may also reduce the precisionrequired, depending upon how crowded the gesture space is near thegesture under consideration. Mapping the actual motion to a gesture mayalso include accessing gesture database 382.

At step 384, the gesture is mapped to a function for the device. Thisstep may include accessing a function mapping database 386 which mayinclude correlation between gestures and functions. Different users mayhave different mappings of gestures to functions and differentuser-created functions. Thus, function mapping database 386 may alsoinclude user-specific mapping instructions or characteristics,user-created functions (e.g., macros and/or phone numbers) and any otherfunction information which may be applicable to mapping a particulargesture to one or more functions. In some embodiments, gestures may bemapped to individual keystrokes. User identities 381 may also beaccessed in this step. In addition, device context information 388 mayalso be accessed and utilized in mapping the gesture, which may includeenvironmental model information 389, application in focus information390 and device state information 391, such as time and date information,location information, battery condition and mode information (e.g.,silent mode). At steps 392, the device performs the appropriately-mappedone or more function(s), such as Function 1 at step 392 a, Function 2 atstep 392 b or Function 3 at step 392 c.

As discussed above, in particular embodiments handheld device 10 maycomprise a cellular phone with many of the capabilities describedherein. For example, cellular phones with motion input capabilities mayuse motion input to flatten menus as discussed above. The cellular phonemay detect device states and environments, such as free fall or thecellular phone being face down or face up to map to behaviors such asmute, speaker phone and power-off. Other detection of device states mayinclude detecting that the phone is being held to disengage mute orspeakerphone states. The cellular phone may utilize gestures to controldialing (e.g., through gestural speed dial) or to lock/unlock a keypadof the device. For example, the device may be moved in a clockwisecircle to dial home, a counterclockwise circle to dial work and in theshape of a heart to dial a significant other. Users may also be able toprogram the cellular phone to customized gestural mappings.

In particular embodiments handheld device 10 may comprise a digitalcamera utilizing motion input for at least some of the functionsdescribed herein. For example, digital cameras with motion inputcapabilities may use motion input to flatten menus as discussed above.Motion may also be used to allow a user to zoom in (and back out) onstill photos or video to examine it more closely for smoother and moreintuitive functionality. Motion may be used to zoom in and out of anumber of thumbnails of photographs or video clips so that it is easy toselect one or more to review. Virtual desktops may be used to reviewmany thumbnails of many digital photos or video clips or to review manydigital photos or video clips by translating the camera or usinggestural input. Gestures and simple motions may be used alone or incombination with other interface mechanisms to modify various settingson digital still and video cameras, such as flash settings, type offocus and light sensing mode. Moreover, free fall may be detected toinduce the camera to protect itself in some way from damage in animpending collision. Such protection may include dropping power fromsome or all parts of the camera, closing the lens cover and retractingthe lens.

In particular embodiments handheld device 10 may comprise a digitalwatch utilizing motion input for at least some of the functionsdescribed herein. For example, digital watches with motion inputcapabilities may use motion input to flatten menus as discussed above.In some embodiments, the tapping of the watch or particular gestures maybe used to silence the watch. Other functions may also be accessedthrough taps, rotations, translations and other more complex gestures.These functions may include changing time zones, setting the watch(e.g., setting the time and other adjustable settings), changing modes(e.g., timers, alarms, stopwatch), activating the backlight, using astopwatch (e.g., starting, stopping and splitting the stopwatch) andstarting and stopping other timers.

In some embodiments, motion detection may be separate from a display.For example, a display may be worn on glasses or contacts, and otherparts of the handheld device may be dispersed across a user's body suchthat the display may not be part of the same physically component as themotion input device or component.

As discussed above, particular figures illustrate various methods,flowcharts and processes which may be performed in particularembodiments. It should be understood that steps may be performed in anyorder, and steps from a particular method, flowchart or process may becombined with other methods, flowcharts or processes or other steps fromthe same method, flowchart or process in various embodiments withoutdeparting from the scope of the invention.

Although the present invention has been described in detail withreference to particular embodiments, it should be understood thatvarious other changes, substitutions, and alterations may be made heretowithout departing from the spirit and scope of the present invention.For example, although the present invention has been described withreference to a number of elements included within handheld device 10,these elements may be combined, rearranged or positioned in order toaccommodate particular architectures or needs. In addition, any of theseelements may be provided as separate external elements to each otherwhere appropriate. The present invention contemplates great flexibilityin the arrangement of these elements as well as their internalcomponents.

Numerous other changes, substitutions, variations, alterations andmodifications may be ascertained by those skilled in the art and it isintended that the present invention encompass all such changes,substitutions, variations, alterations and modifications as fallingwithin the spirit and scope of the appended claims.

1. A handheld device comprising: a display having a viewable surface and operable to generate an image indicating a currently controlled remote device; a gesture database maintaining a plurality of remote command gestures, each remote command gesture defined by a motion of the device with respect to a first position of the handheld device; a gesture mapping database comprising a mapping of each of the remote command gestures to an associated command for controlling operation of the remote device; a motion detection module operable to detect motion of the handheld device within three dimensions and to identify components of the motion in relation to the viewable surface; a control module operable to track movement of the handheld device using the motion detection module, to compare the tracked movement against the remote command gestures to determine a matching gesture, and to identify the one of the commands corresponding to the matching gesture; and a wireless interface operable to transmit the identified command to a remote receiver for delivery to the remote device.
 2. The handheld device of claim 1, wherein the remote receiver comprises a wireless interface of the remote device.
 3. The handheld device of claim 1, wherein the remote receiver comprises an element of a public wireless telephone network.
 4. The handheld device of claim 1, wherein the remote device comprises audio/visual equipment.
 5. The handheld device of claim 4, wherein the identified command controls output of the audio/visual equipment.
 6. The handheld device of claim 1, wherein the wireless interface is further operable to transmit the matching gesture to the remote receiver for delivery to the remote device.
 7. The handheld device of claim 1, further comprising: a first accelerometer operable to detect acceleration along a first axis; a second accelerometer operable to detect acceleration along a second axis, the second axis perpendicular to the first axis; and a third accelerometer operable to detect acceleration along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis; and wherein: the gesture database further defines each of the remote command gestures using a sequence of accelerations; the motion detection module is further operable to detect motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and the control module is further operable to match the accelerations measured by the motion detection module against gesture definitions in the gesture database to identify particular ones of the remote command gestures.
 8. A method for remotely controlling devices comprising: generating, on a viewable surface of a handheld device, an image indicating a currently controlled remote device; maintaining a gesture database comprising a plurality of remote command gestures, each remote command gesture defined by a motion of the device with respect to a first position of the handheld device; maintaining a gesture mapping database comprising a mapping of each of the remote command gestures to an associated command for controlling operation of the remote device; tracking movement of the handheld device in relation to the viewable surface; comparing the tracked movement against the remote command gestures to determine a matching gesture; identifying the one of the commands corresponding to the matching gesture; and transmitting the identified command to a remote receiver for delivery to the remote device.
 9. The method of claim 8, wherein the remote receiver comprises a wireless interface of the remote device.
 10. The method of claim 8, wherein the remote receiver comprises an element of a public wireless telephone network.
 11. The method of claim 8, wherein the remote device comprises audio/visual equipment.
 12. The method of claim 11, wherein the identified command controls output of the audio/visual equipment.
 13. The method of claim 8, wherein the gesture database further defines each of the remote command gestures using a sequence of accelerations; the method further comprising: detecting acceleration of the handheld device along a first axis; detecting acceleration of the handheld device along a second axis, the second axis perpendicular to the first axis; and detecting acceleration of the handheld device along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis; detecting motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and matching the accelerations against gesture definitions in the gesture database to identify potential indicated ones of the remote command gestures.
 14. Logic for controlling a handheld device, the logic embodied in a computer readable medium and operable when executed to perform the steps of: generating, on a viewable surface of a handheld device, an image indicating a currently controlled remote device; maintaining a gesture database comprising a plurality of remote command gestures, each remote command gesture defined by a motion of the device with respect to a first position of the handheld device; maintaining a gesture mapping database comprising a mapping of each of the remote command gestures to an associated command for controlling operation of the remote device; tracking movement of the handheld device in relation to the viewable surface; comparing the tracked movement against the remote command gestures to determine a matching gesture; identifying the one of the commands corresponding to the matching gesture; and transmitting the identified command to a remote receiver for delivery to the remote device.
 15. The logic of claim 14, wherein the remote receiver comprises a wireless interface of the remote device.
 16. The logic of claim 14, wherein the remote receiver comprises an element of a public wireless telephone network.
 17. The logic of claim 14, wherein the remote device comprises audio/visual equipment.
 18. The logic of claim 17, wherein the identified command controls output of the audio/visual equipment.
 19. The logic of claim 14, wherein the gesture database further defines each of the remote command gestures using a sequence of accelerations; the logic further operable when executed to perform the steps of: detecting acceleration of the handheld device along a first axis; detecting acceleration of the handheld device along a second axis, the second axis perpendicular to the first axis; and detecting acceleration of the handheld device along a third axis, the third axis perpendicular to the first axis and perpendicular to the second axis; detecting motion of the device using accelerations measured by the first accelerometer, the second accelerometer, and the third accelerometer; and matching the accelerations against gesture definitions in the gesture database to identify potential indicated ones of the remote command gestures.
 20. A motion controlled handheld device comprising: means for generating, on a viewable surface of a handheld device, an image indicating a currently controlled remote device; means for maintaining a gesture database maintaining a plurality of remote command gestures, each remote command gesture defined by a motion of the device with respect to a first position of the handheld device; means for maintaining a gesture mapping database comprising a mapping of each of the remote command gestures to an associated command for controlling operation of the remote device; means for tracking movement of the handheld device in relation to the viewable surface; means for comparing the tracked movement against the remote command gestures to determine a matching gesture; means for identifying the one of the commands corresponding to the matching gesture; and means for transmitting the identified command to a remote receiver for delivery to the remote device. 